diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 5751527..fdd42b2 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -1,6 +1,6 @@ # Copilot Instructions for multiplex -*multiplex* is a modular framework for prototyping LLM-based mutation testing. One run mutates a single Java method (located via tree-sitter), generates mutants with an LLM, splices each mutant back into the source file, and runs the target project's tests to see which mutants are killed. +*multiplex* is a modular framework for prototyping LLM-based mutation testing. One run mutates a single method/function — Java or Python, set by `project.language` and located via tree-sitter — generates mutants with an LLM, splices each mutant back into the source file, and runs the target project's tests to see which mutants are killed. ## Read the docs before reading source @@ -26,7 +26,8 @@ uv run ./multiplex ./path/to/config.yml # run the tool - **Import convention**: the tool runs as a directory (`uv run ./multiplex`), so `multiplex/` itself is on `sys.path`. Modules inside `multiplex/` import each other **without** the package prefix (`from model import Model`, `from util.io import ...`); tests import **with** it (`from multiplex.checks... import ...`). Match the style of the file being edited. - Adding a mutant-generation approach touches three places: a new `multiplex/approach//` package with `controller.py`, an `APPROACH_PROMPT_KEYS` entry in `multiplex/prompts.py`, and a dispatch branch in `multiplex/__main__.py`. Only the selected approach's `system_prompts` keys are required; `resolve_prompts` validates the approach and its keys up front, raising `SystemExit` before any destructive step. -- Mutant files must be complete replacement methods written to `output/-mutants/mutant_N.java`; they are spliced verbatim over the original method's byte range. -- tree-sitter versions are pinned (`tree-sitter==0.23.2`, `tree-sitter-java==0.23.5`); do not bump them casually — the parsing code depends on that API. -- A runnable end-to-end example lives in `examples/` (self-contained Maven project, `basic` approach, `mvn` backend): `uv run multiplex ./examples/config.yml`. +- Mutant files must be complete replacement methods written to `output/-mutants/mutant_N.` (`.java`/`.py`, from the `LanguageSpec`); they are spliced verbatim over the original method's byte range. +- The source language is set by `project.language` (default `java`); a `languages.LanguageSpec` (grammar, node types, extension, fence, prompt noun) is resolved once in `__main__.py` and threaded into `extract_method`, each approach's `main`, the checks, and the execution backend. Adding a language = a `_REGISTRY` entry in `multiplex/languages/__init__.py` (+ prompts, example, and an llmorpheus query). See `docs/EXTENDING.md`. +- tree-sitter versions are pinned (`tree-sitter==0.23.2`, `tree-sitter-java==0.23.5`, `tree-sitter-python` 0.23.x); do not bump them casually — the parsing code depends on that API. +- Runnable end-to-end examples live in `examples/`: Java (`config-java.yml`, Maven project, `mvn` backend) and Python (`config-python.yml`, pytest project, `pytest` backend). `uv run ./multiplex ./examples/config-java.yml`. - Keep pure logic (parsing, splicing, checks) separate from `model.make_request` call sites — LLM calls are not mocked in tests. diff --git a/AGENTS.md b/AGENTS.md index 844bcaa..1ffa12d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,7 +2,7 @@ Guidance for AI coding agents working in this repository. -*multiplex* is a modular framework for prototyping LLM-based mutation testing. One run mutates a single Java method (located via tree-sitter), generates mutants with an LLM, splices each mutant back into the source file, and runs the target project's tests to see which mutants are killed. +*multiplex* is a modular framework for prototyping LLM-based mutation testing. One run mutates a single method/function — Java or Python, set by `project.language` and located via tree-sitter — generates mutants with an LLM, splices each mutant back into the source file, and runs the target project's tests to see which mutants are killed. ## Documentation map @@ -11,7 +11,7 @@ Task-scoped docs live in `docs/` — start at [docs/README.md](docs/README.md) a - [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) — pipeline, approaches, execution backends, output artifacts. - [docs/MODULE_REFERENCE.md](docs/MODULE_REFERENCE.md) — every function's signature, behavior, and side effects. - [docs/CONFIG.md](docs/CONFIG.md) — full `config.yml` schema. -- [docs/EXTENDING.md](docs/EXTENDING.md) — checklists for adding approaches, execution backends, or LLM providers. +- [docs/EXTENDING.md](docs/EXTENDING.md) — checklists for adding approaches, execution backends, languages, or LLM providers. - [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) — setup, conventions, CI, known bugs/gotchas. ## Commands @@ -30,4 +30,4 @@ uv run ./multiplex ./path/to/config.yml # run the tool - Pipeline entry point and dispatch: `multiplex/__main__.py`. Adding an approach touches three places (new `approach//controller.py` package + `APPROACH_PROMPT_KEYS` entry in `multiplex/prompts.py` + dispatch branch). Only the selected approach's `system_prompts` keys are required; `resolve_prompts` validates them (and the approach name) up front, raising `SystemExit` before any destructive step. See `docs/EXTENDING.md`. - `/output/` is wiped at the start of every run; the target source file is edited in place and restored from a `.orig` backup. - tree-sitter versions are pinned; do not bump them casually. -- A runnable example lives in `examples/`: `uv run multiplex ./examples/config.yml` (self-contained Maven project, needs `mvn` + a JDK + a running Ollama). See `docs/DEVELOPMENT.md` § Example. +- Runnable examples live in `examples/`: `uv run ./multiplex ./examples/config-java.yml` (Java, Maven project — needs `mvn` + a JDK) and `uv run ./multiplex ./examples/config-python.yml` (Python, pytest backend). Both need a running Ollama. See `docs/DEVELOPMENT.md` § Example. diff --git a/README.md b/README.md index fee69ca..9af123e 100644 --- a/README.md +++ b/README.md @@ -69,20 +69,21 @@ Once configured and modules are set up, users can run *multiplex* using the foll uv run /path/to/multiplex ./path/to/config.yml ``` -#### Try the bundled example -A self-contained example (a small Maven project mutated with the `basic` -approach) is included. With `mvn` + a JDK on your `PATH` and a running Ollama -(`ollama pull gpt-oss:20b`, or edit `llm.model` in the config), run from the -repo root: +#### Try the bundled examples +Self-contained examples are included for both supported languages, each mutating +a small project with the `basic` approach. With a running Ollama +(`ollama pull gpt-oss:20b`, or edit `llm.model` in the config), run from the repo +root: ```bash -uv run multiplex ./examples/config.yml +uv run ./multiplex ./examples/config-java.yml # Java (needs mvn + a JDK on PATH) +uv run ./multiplex ./examples/config-python.yml # Python (uses pytest via uv run) ``` -Results are written to `examples/project/example/output/basic-mutants/` +Results are written to the project's `output/-mutants/` (`mutant_summary.csv`). See [`examples/README.md`](examples/README.md) for details. ## 🧩 Existing Modules -*multiplex* currently includes five mutant generation modules and two execution and evaluation modules: +*multiplex* currently includes five mutant generation modules and three execution and evaluation modules: **Mutant Generation Modules:** - HAZOP @@ -94,6 +95,12 @@ details. **Execution and Evaluation Modules** - Maven - Defects4J (enables users to target specific bugs in the [Defects4J](https://github.com/rjust/defects4j) dataset) +- Pytest (for Python projects) + +**Languages:** Java (default) and Python, selected per run with `project.language` +in the config. All five generation modules and the equivalence/compilable checks +are language-aware. See [`docs/EXTENDING.md`](docs/EXTENDING.md) § Add a language +for how to add another. ## 🏗️ Adding Modules diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index d17a447..62a14e9 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -1,8 +1,10 @@ # Architecture *multiplex* is a modular framework for prototyping LLM-based mutation testing. -One run mutates **one Java method** in a target project and evaluates the -generated mutants against that project's test suite. +One run mutates **one method/function** in a target project and evaluates the +generated mutants against that project's test suite. The source language is set +by `project.language` (`java` — the default — or `python`); see the language +registry below. ## Pipeline @@ -12,25 +14,26 @@ Orchestrated top-to-bottom by `multiplex/__main__.py`: config.yml │ ▼ -1. Load YAML config; read all 9 system_prompts keys into a dict +1. Load YAML config; resolve the approach's system_prompts and the language + (languages.get_language(project.language), default "java") │ ▼ 2. Back up target source file to .orig Delete and recreate /output/ ← destroys previous results │ ▼ -3. util/extract_method.py (tree-sitter) - Find method by name + start line - → writes output/original_method.java +3. util/extract_method.py (tree-sitter, grammar from the LanguageSpec) + Find method/function by name + start line + → writes output/original_method. (.java / .py) → returns (start_byte, end_byte) offsets into the source file │ ▼ -4. approach//controller.main(model, output_path, prompts) +4. approach//controller.main(model, output_path, prompts, language) One or more LLM calls (via model.Model / LiteLLM) - → writes mutant files to output/-mutants/mutant_N.java + → writes mutant files to output/-mutants/mutant_N. │ ▼ -5. execute/{maven,defects4j}.run_mutants(...) +5. execute/{maven,defects4j,pytest_runner}.run_mutants(...) For each mutant file: a. restore source file from .orig backup b. util/rewrite_method.py splices mutant text into the file @@ -51,7 +54,7 @@ changes the file between those steps invalidates the offsets. ## Mutant-generation approaches (`multiplex/approach/`) Each approach is a package with a `controller.py` exposing -`main(model, output_dir, prompts)`. Dispatch is an if/elif chain in +`main(model, output_dir, prompts, language)`. Dispatch is an if/elif chain in `__main__.py` keyed on `mutation.approach`. | Approach | Strategy | LLM calls | @@ -62,6 +65,11 @@ Each approach is a package with a `controller.py` exposing | `mutahunter` | Single call with AST + line-numbered source; LLM returns YAML of line-level mutations spliced in locally (prompts not bundled — licensing) | 1 | | `llmorpheus` | tree-sitter query finds mutation sites (conditions, loop headers, call args), each replaced by ``; LLM proposes 3 replacements per site | 1 per placeholder | +All five approaches are language-aware via the `language` argument (file +extension, code fence, prompt noun). llmorpheus additionally selects its +mutation-site query per language from `MUTATION_QUERIES` in +`approach/llmorpheus/placeholders.py`. + Intermediate artifacts are files in `output/` — approaches communicate between their own steps via files, not in-memory state (see artifact table below). @@ -78,8 +86,24 @@ Selected by `project.runtool`: `mvn -f clean test`; a mutant survives if the build passes (`BUILD SUCCESS`). Baselines the original first (aborts if its tests fail), then per mutant does the same equivalence → rewrite → compilable → test → - summary flow as the Defects4J backend. Used by the runnable example under + summary flow as the Defects4J backend. Used by the runnable Java example under `examples/` (see DEVELOPMENT.md § Example). +- `pytest` → `pytest_runner.py` — self-contained backend for Python projects. + Runs `sys.executable -m pytest -q ` (pytest under multiplex's own + interpreter); a mutant survives if pytest exits 0 (all tests pass). Same + baseline → per-mutant flow as the Maven backend. Drives the runnable Python + example (`examples/config-python.yml`). Named `pytest_runner` so it does not + shadow the installed `pytest` package. + +## Language registry (`multiplex/languages/`) + +`get_language(project.language)` returns a `LanguageSpec` (grammar, definition +node types, comment node types, extension, code-fence tag, prompt noun, +mutahunter label). It is resolved once in `__main__.py` and threaded into +`extract_method`, every approach's `main`, the checks, and the execution +backend — so the pipeline is language-agnostic and adding a language is a +registry entry plus prompts and an example (see docs/EXTENDING.md § Add a +language). `project.language` defaults to `java`. ## Output artifacts @@ -87,12 +111,12 @@ Everything lands under `/output/` (wiped at the start of each run): | Artifact | Written by | |----------|-----------| -| `original_method.java` | extract_method (step 3); read by every approach | +| `original_method.` | extract_method (step 3); read by every approach (`.java`/`.py`) | | `hazop-descriptions.txt`, `hazop-mutated-descriptions.txt` | hazop chain steps | | `control_diagram.txt`, `ucas.csv` | stpa chain steps | -| `placeholders/{N_placeholder.java, N_orig.java, placeholders.json}` | llmorpheus site finder | -| `-mutants/mutant_N.java` | every approach's final step | -| `-mutants/mutant_summary.csv` | defects4j backend; columns `MUTANT, EQUIVALENCE, COMPILABLE, SURVIVES` | +| `placeholders/{N_placeholder., N_orig., placeholders.json}` | llmorpheus site finder | +| `-mutants/mutant_N.` | every approach's final step | +| `-mutants/mutant_summary.csv` | maven/defects4j/pytest backends; columns `MUTANT, EQUIVALENCE, COMPILABLE, SURVIVES` | | `-test/_test.txt` | defects4j backend; per-mutant test output | ## LLM access (`multiplex/model.py`) @@ -104,4 +128,5 @@ in config). Azure endpoints get `AZURE_AI_API_BASE`/`AZURE_AI_API_KEY` set specially. To support another provider, replace or extend `model.py`. LLM responses are treated as fenced code: approaches strip a leading -```` ```java ```` fence and keep everything before the next ```` ``` ```` fence. +```` ``` ```` fence (the tag is `language.fence`, e.g. `java`/`python`) and +keep everything before the next ```` ``` ```` fence. diff --git a/docs/CONFIG.md b/docs/CONFIG.md index 75da8ca..a15633e 100644 --- a/docs/CONFIG.md +++ b/docs/CONFIG.md @@ -2,7 +2,8 @@ *multiplex* takes a single YAML config file: `uv run ./multiplex ./path/to/config.yml`. -Template: [`examples/config.yml`](../examples/config.yml). +Templates: [`examples/config-java.yml`](../examples/config-java.yml), +[`examples/config-python.yml`](../examples/config-python.yml). Only the keys the selected `mutation.approach` needs are required. `project`, `mutation`, and `llm` keys are always required; `system_prompts` keys are @@ -13,11 +14,12 @@ startup with an actionable `SystemExit`, before any output is wiped. | Key | Type | Meaning | |-----|------|---------| -| `projectroot` | path | Root of the Java project under test. `output/` is created (and **wiped every run**) inside it. | -| `filename` | path | The `.java` source file containing the method under test. Backed up to `.orig` during the run. | -| `method` | string | Name of the method (or constructor) to mutate. | -| `line` | int | 1-based line number of the method **name identifier** in `filename` (not annotations above it). Both `method` and `line` must match for extraction to succeed; disambiguates overloads. | -| `runtool` | `mvn` \| `d4j` | Execution backend. `mvn` runs a plain Maven project (`mvn clean test`) and is used by the runnable `examples/` setup; `d4j` targets a Defects4J checkout (needs the `defects4j` CLI + `JDK_11`). | +| `language` | `java` \| `python` | Source language of the file under test. **Optional; defaults to `java`** (existing configs need no change). Selects the tree-sitter grammar, the definition node types extracted, the artifact/mutant file extension, and the code-fence/prompt wording. An unknown value fails fast at startup with a `SystemExit`. | +| `projectroot` | path | Root of the project under test. `output/` is created (and **wiped every run**) inside it. | +| `filename` | path | The source file containing the method/function under test (`.java` or `.py`). Backed up to `.orig` during the run. | +| `method` | string | Name of the method/constructor (Java) or function (Python) to mutate. | +| `line` | int | 1-based line number of the method/function **name identifier** in `filename` (not annotations/decorators above it). Both `method` and `line` must match for extraction to succeed; disambiguates overloads. | +| `runtool` | `mvn` \| `d4j` \| `pytest` | Execution backend. `mvn` runs a plain Maven project (`mvn clean test`); `d4j` targets a Defects4J checkout (needs the `defects4j` CLI + `JDK_11`); `pytest` runs pytest under multiplex's own interpreter (`sys.executable -m pytest `) — a mutant survives if pytest exits 0. `mvn` drives the Java `examples/` setup and `pytest` the Python one. | ## `mutation` diff --git a/docs/DEVELOPMENT.md b/docs/DEVELOPMENT.md index ce17867..3c0623e 100644 --- a/docs/DEVELOPMENT.md +++ b/docs/DEVELOPMENT.md @@ -17,36 +17,46 @@ uv run ./multiplex ./path/to/config.yml # run the tool litellm's `tiktoken` dependency fails to build on 3.14. - Tests must run from the repo root: they reference `tests/resources/...` relatively. -- Running the tool needs a reachable LLM endpoint and a target Java project; - `d4j` runs additionally need the `defects4j` CLI (expected under - `./defects4j/framework/bin`) and a `JDK_11` env var pointing at a JDK 11 home. +- Running the tool needs a reachable LLM endpoint and a target project (Java or + Python, per `project.language`). `mvn` runs need Maven + a JDK; `pytest` runs + need pytest for the active interpreter; `d4j` runs additionally need the + `defects4j` CLI (expected under `./defects4j/framework/bin`) and a `JDK_11` env + var pointing at a JDK 11 home. CI (GitHub Actions, on push/PR to `main`): `test.yml` runs `uv run pytest tests` on Python 3.13; `ruff.yml` runs `uv run ruff check`. ## Example -A self-contained end-to-end example lives under `examples/`: +Self-contained end-to-end examples live under `examples/`: ```bash -uv run multiplex ./examples/config.yml # from the repo root +uv run ./multiplex ./examples/config-java.yml # Java (from the repo root) +uv run ./multiplex ./examples/config-python.yml # Python ``` -- Target: `examples/project/example/` — a tiny Maven project with one method +- Target: `examples/project/java-example/` — a tiny Maven project with one method (`com.example.Classifier.classify`) and JUnit tests pinning its behavior. -- `examples/config.yml` uses the `basic` approach and the `mvn` runtool. +- `examples/config-java.yml` uses the `basic` approach and the `mvn` runtool. - Prerequisites: `mvn` + a JDK 11+ on PATH (the project targets Java 11; if `mvn` picks up an older JDK via `JAVA_HOME` the compile fails with "release version 11 not supported"), and a running Ollama serving the model named in `llm.model` (default `gpt-oss:20b`; point it at any model you have, e.g. `ollama pull gpt-oss:20b`). Any LiteLLM-supported endpoint works if you edit `llm`. -- Output lands in `examples/project/example/output/basic-mutants/` +- Output lands in `examples/project/java-example/output/basic-mutants/` (`mutant_N.java` plus `mutant_summary.csv`). Run artifacts (`output/`, `*.orig`, Maven `target/`) are git-ignored. - The `basic` approach makes 10 LLM calls and the `mvn` backend runs the test suite once per compilable mutant, so a full run takes a few minutes (longer on a slow/local model). +- Python counterpart: `examples/config-python.yml` (`project.language: python`, + `pytest` runtool) mutates `classify` in + `examples/project/python-example/classifier.py` and evaluates each mutant with + pytest (run under multiplex's own interpreter). Same output layout + (`output/basic-mutants/mutant_N.py` + + `mutant_summary.csv`). Needs pytest (already provided by `uv run`) and the same + Ollama/LLM endpoint. ## Import convention (critical) @@ -65,15 +75,17 @@ same process; keep the two worlds separate. ## Conventions -- tree-sitter is the universal Java analysis tool (extraction, compilability, - equivalence, placeholder finding). Versions are pinned - (`tree-sitter==0.23.2`, `tree-sitter-java==0.23.5`); the code depends on that - API — do not bump casually. +- tree-sitter is the universal source-analysis tool (extraction, compilability, + equivalence, placeholder finding); the grammar comes from the run's + `languages.LanguageSpec`. Versions are pinned (`tree-sitter==0.23.2`, + `tree-sitter-java==0.23.5`, `tree-sitter-python` 0.23.x); the code depends on + that API — do not bump casually. - Approaches communicate between their own steps via files in `output/`, not in-memory state; artifact names are prefixed with the approach name. -- LLM code responses are unfenced by stripping a leading ```` ```java ```` - fence and truncating at the next ```` ``` ```` fence - (`removeprefix` + `split`, see EXTENDING.md for the exact two lines). +- LLM code responses are unfenced by stripping a leading ```` ``` ```` + fence (the tag is `language.fence`, e.g. `java`/`python`) and truncating at the + next ```` ``` ```` fence (`removeprefix` + `split`, see EXTENDING.md for the + exact two lines). - Mutant files are named `mutant_.java` in `output/-mutants/`. - Status/progress reporting is `print()`-based throughout (no logging config, one `logging.warning` in extract_method). diff --git a/docs/EXTENDING.md b/docs/EXTENDING.md index 5631836..87755bd 100644 --- a/docs/EXTENDING.md +++ b/docs/EXTENDING.md @@ -12,21 +12,25 @@ the multi-prompt-chain template. 1. **Create the package** `multiplex/approach//` with `__init__.py` and `controller.py` exposing: ```python - def main(model, output_dir, prompts): + def main(model, output_dir, prompts, language): ... ``` - `model` is a `model.Model`; call `model.make_request(messages)` with OpenAI-style message dicts. + - `language` is a `languages.LanguageSpec` (resolved in `__main__.py`); use it + for anything language-specific — `language.extension`, `language.fence`, + `language.noun` — so your approach works for every registered language. - Read the method under test via - `from approach.util import get_method_under_test`. + `from approach.util import get_method_under_test` — call it as + `get_method_under_test(output_dir, language)`. - Pass intermediate results between steps as files in `output_dir` (convention: prefix them with your approach name). - **Required output**: write final mutants to - `Path(output_dir, "-mutants/") / f"mutant_{count}.java"`. Each file - must contain the complete replacement method body (it is spliced verbatim - over the original method's byte range). Strip LLM code fences: + `Path(output_dir, "-mutants/") / f"mutant_{count}{language.extension}"`. + Each file must contain the complete replacement method body (it is spliced + verbatim over the original method's byte range). Strip LLM code fences: ```python - mutant = mutant.removeprefix("```java") + mutant = mutant.removeprefix("```" + language.fence) mutant = mutant.split("```", 1)[0] ``` 2. **Register the prompt key(s)**: add a `"": [...]` entry to @@ -34,10 +38,10 @@ the multi-prompt-chain template. `system_prompts` keys your approach reads. Only the selected approach's keys are required at runtime, so users running other approaches need not define yours (and vice versa). Document the keys in `docs/CONFIG.md`; adding them to - `examples/config.yml` (a commented stub is fine) is optional but helpful. + `examples/config-java.yml` (a commented stub is fine) is optional but helpful. 3. **Register the dispatch branch**: add an `elif config['mutation']['approach'] == "":` branch in `multiplex/__main__.py` calling - `.main(model, output_path, prompts)`, plus the corresponding + `.main(model, output_path, prompts, language)`, plus the corresponding `import approach..controller as `. 4. The `-mutants` directory name must equal the `mutation.approach` config value — the execution backends reconstruct the path as @@ -51,24 +55,72 @@ Model on `multiplex/execute/defects4j.py` or `multiplex/execute/maven.py` 1. Create `multiplex/execute/.py` exposing: ```python def run_mutants(project_root, original_file, output_path, - method_start_byte, method_end_byte, duplicate, approach): + method_start_byte, method_end_byte, duplicate, approach, + language): ``` + `language` is a `languages.LanguageSpec` — use it for the artifact path and + the checks below. 2. The expected loop, per mutant file in `output/-mutants/`: - restore the pristine source: `shutil.copy2(duplicate, original_file)` (guard on `duplicate.exists()`); - `rewrite_method(original_file, method_start_byte, method_end_byte, mutant_path)` — note: **4 arguments**; - - `check_mutant_equivalent(mutant_path, output/original_method.java)` and - `check_mutant_compilable(original_file)` from `checks/`; + - `check_mutant_equivalent(mutant_path, + language.original_method_path(output_path), language)` and + `check_mutant_compilable(original_file, language)` from `checks/`; - if compilable, run the project's tests with your build tool and decide survived/killed; - collect rows and finish with `write_mutant_summary(mutants_dir, rows)` (header row: `["MUTANT", "EQUIVALENCE", "COMPILABLE", "SURVIVES"]`). -3. Register it in `multiplex/__main__.py` under a new `project.runtool` value. +3. Register it in `multiplex/__main__.py` under a new `project.runtool` value + (pass `language` through to `run_mutants`). 4. Useful pattern from defects4j: run the unmutated project first as a baseline (abort if it fails), and use a multiple of its wall-clock time as the per-mutant test timeout to catch infinite-loop mutants. +## Add a language + +The pipeline is language-agnostic: the parser, checks, approaches and execution +backends all take a `languages.LanguageSpec` resolved once in `__main__.py` from +the `project.language` config value. Java and Python ship in +`multiplex/languages/__init__.py`. + +To add a language: + +1. **Add the tree-sitter grammar** dependency (e.g. `tree-sitter-`) to + `pyproject.toml` and add a `LanguageSpec` entry to `_REGISTRY` in + `multiplex/languages/__init__.py`: + ```python + "": LanguageSpec( + name="", + extension=".", + ts_language=Language(ts_.language()), + def_node_types=frozenset({...}), # nodes that denote a definition + comment_node_types=frozenset({...}), # ignored by the equivalence check + fence="", # ``` fence tag + noun=" function", # prompt wording + label="", # mutahunter language string + ), + ``` + `def_node_types` are the node types `extract_method` matches (their child + `identifier` is compared to `project.method`); `comment_node_types` are + dropped when comparing ASTs for syntactic equivalence. Inspect a grammar with + a few lines of tree-sitter to find the right node types. +2. **Provide a prompt set** for the language in the config's `system_prompts` + (see `examples/config-python.yml`). Prompt *keys* are shared across languages; + only the wording differs. mutahunter prompts remain user-supplied. +3. **Add a mutation-site query for llmorpheus** if you want that approach: + add a `"": ` entry to `MUTATION_QUERIES` in + `multiplex/approach/llmorpheus/placeholders.py` (keyed by `language.name`). + The other four approaches work with no per-approach changes. +4. **Pick an execution backend**: reuse an existing `project.runtool` + (`mvn`/`d4j`/`pytest`) or add one (see the section above). `pytest_runner.py` + is the template for a script/test-command backend. +5. **Add an end-to-end example** under `examples/` (a small project + a config + with `project.language: `), mirroring `examples/project/python-example/`. + +Existing configs are unaffected: `project.language` defaults to `java`. + ## Add / change the LLM provider `multiplex/model.py` wraps LiteLLM, so most providers work by editing only the diff --git a/docs/MODULE_REFERENCE.md b/docs/MODULE_REFERENCE.md index 4f4a762..1983ea9 100644 --- a/docs/MODULE_REFERENCE.md +++ b/docs/MODULE_REFERENCE.md @@ -6,6 +6,27 @@ Signatures are exact; use this instead of opening source files. Note on imports: inside `multiplex/` modules import each other without the package prefix (e.g. `from util.io import write_to_file`). See DEVELOPMENT.md. +Most pipeline functions take a `language` argument — a `languages.LanguageSpec` +resolved once in `__main__.py` from `project.language` (default `java`). + +## languages/__init__.py + +```python +@dataclass(frozen=True) +class LanguageSpec: + name: str; extension: str; ts_language: Language + def_node_types: frozenset; comment_node_types: frozenset + fence: str; noun: str; label: str + def original_method_path(self, output_dir) -> Path # output_dir/original_method + +get_language(name=None) -> LanguageSpec # default "java"; SystemExit on unknown +DEFAULT_LANGUAGE = "java" +``` + +- `_REGISTRY` holds the `java` and `python` specs. `name` matching is + case-insensitive. Raises `SystemExit` (actionable, lists valid languages) for + an unknown name — same fail-fast contract as `prompts.resolve_prompts`. + ## model.py ```python @@ -53,15 +74,16 @@ reset_source_code(duplicate_file_path, filename) ## util/extract_method.py ```python -extract_method_from_file(file_path, method_name, output_dir, start_line) +extract_method_from_file(file_path, method_name, output_dir, start_line, language) -> (start_byte, end_byte) | None ``` -- tree-sitter walk for a `method_declaration`/`constructor_declaration` whose - identifier text == `method_name` **and** whose identifier is on line - `start_line` (1-based). Both must match — `start_line` is the line of the - method *name*, not of annotations above it. -- Writes the method source to `output_dir/original_method.java`. +- tree-sitter walk (grammar from `language`) for a node in + `language.def_node_types` whose identifier text == `method_name` **and** whose + identifier is on line `start_line` (1-based). Both must match — `start_line` is + the line of the method/function *name*, not of annotations/decorators above it. +- Writes the method source to `language.original_method_path(output_dir)` + (`output_dir/original_method.`). - Returns `None` if not found (logs a warning). The caller in `__main__.py` checks for `None` and exits with a clear `SystemExit` naming `project.method`/`project.line`. @@ -90,36 +112,40 @@ add_mutant_to_method(numbered_src, mutant, line_number) -> str ## checks/compilable.py ```python -check_mutant_compilable(filename) -> bool +check_mutant_compilable(filename, language) -> bool ``` -- **Not a real compile.** tree-sitter parses the file and returns False if any - `ERROR` or missing node exists. Catches syntax errors only — type errors, - missing symbols, etc. pass. +- **Not a real compile.** tree-sitter (grammar from `language`) parses the file + and returns False if any `ERROR` or missing node exists. Catches syntax errors + only — type errors, missing symbols, etc. pass. ## checks/syntactic_equivalence.py ```python -check_mutant_equivalent(mutant_filename, original_filename) -> bool +check_mutant_equivalent(mutant_filename, original_filename, language) -> bool ``` -- Serializes both files' tree-sitter ASTs to strings, dropping comment nodes; - True iff the strings are identical. Detects mutants that only change - comments/formatting. Prints a `SequenceMatcher` similarity ratio as a side - effect (not used in the decision). +- Serializes both files' tree-sitter ASTs to strings, dropping + `language.comment_node_types`; True iff the strings are identical. Detects + mutants that only change comments/formatting. Prints a `SequenceMatcher` + similarity ratio as a side effect (not used in the decision). ## approach/util.py ```python -get_method_under_test(output_dir) -> str # reads output_dir/original_method.java; FileNotFoundError if absent +get_method_under_test(output_dir, language) -> str # reads language.original_method_path(output_dir); FileNotFoundError if absent ``` ## approach/*/controller.py (all five) ```python -main(model, output_dir, prompts) # prompts: the full dict built in __main__.py +main(model, output_dir, prompts, language) # prompts: the full dict built in __main__.py ``` +Each approach's `generate_code` uses `language` for the mutant file extension, +the ```` ``` ````-fence tag it strips, and the prompt noun +(`Java method`/`Python function`). + Each controller just calls its package's step functions in order with the relevant `prompts[...]` key(s): @@ -138,23 +164,25 @@ relevant `prompts[...]` key(s): `mutants: [{mutated_code, line_number}, ...]`, spliced locally via `util.parser.add_mutant_to_method`. The user-prompt template is intentionally gutted (licensing) — users must fill in Mutahunter's own prompt text. -- `llmorpheus`: `placeholders.create_placeholders` — tree-sitter query captures - if/while/do/switch conditions, for-loop parts, enhanced-for parts, loop - headers, and method-call names/receivers/args; each capture produces - `placeholders/N_placeholder.java` (method with `` substituted), - `placeholders/N_orig.java` (original fragment), and an entry in +- `llmorpheus`: `placeholders.create_placeholders` — the tree-sitter query for + `language.name` (from `MUTATION_QUERIES`) captures mutation sites (Java: + if/while/do/switch conditions, for-loop parts, loop headers, call + names/receivers/args; Python: if/while conditions, for target/iterable, + comparison/boolean/binary operators, call function/args); each capture produces + `placeholders/N_placeholder.` (method with `` substituted), + `placeholders/N_orig.` (original fragment), and an entry in `placeholders/placeholders.json` (`{tag, start_byte, end_byte, text}`). `code_generator.generate_code` — one request per placeholder asking for 3 replacements ("Option 1/2/3" fenced blocks, extracted with the regex ```` \n```\n(.*?)\n``` ```` under `re.DOTALL`); each replacement spliced into the method at the placeholder's byte offsets → - `llmorpheus-mutants/mutant_N.java`. + `llmorpheus-mutants/mutant_N.`. ## execute/defects4j.py ```python run_mutants(project_root, original_file, output_path, - method_start_byte, method_end_byte, duplicate, approach) + method_start_byte, method_end_byte, duplicate, approach, language) ``` - `_execute(project_root, file=None, approach=None, timer=None) -> bool`: @@ -173,7 +201,7 @@ run_mutants(project_root, original_file, output_path, ```python run_mutants(project_root, filename, output_path, - method_start_byte, method_end_byte, duplicate, approach) + method_start_byte, method_end_byte, duplicate, approach, language) ``` - `_execute(project_root) -> bool`: runs `mvn -f clean test`; @@ -184,4 +212,21 @@ run_mutants(project_root, filename, output_path, project (raises `IOError` if its tests fail), then per mutant: restore from backup → equivalence check → rewrite → compilable check → (if compilable) run tests → append `[name, equivalent, compilable, survives]` → write - `mutant_summary.csv`. Drives the runnable example under `examples/`. + `mutant_summary.csv`. Drives the runnable Java example under `examples/`. + +## execute/pytest_runner.py + +```python +run_mutants(project_root, original_file, output_path, + method_start_byte, method_end_byte, duplicate, approach, language) +``` + +- `_execute(project_root) -> bool`: runs `sys.executable -m pytest -q + ` (argv list, no shell) — pytest runs under multiplex's own + interpreter, not a `python` resolved from PATH. True iff pytest exits 0 (all + tests pass → mutant survived), False otherwise (mutant killed, incl. + collection/syntax errors). If pytest is not installed for that interpreter, + raises a `SystemExit`. +- `run_mutants`: identical baseline → per-mutant flow as the Maven backend. + Drives the runnable Python example (`examples/config-python.yml`). Module named + `pytest_runner` so importing it does not shadow the installed `pytest` package. diff --git a/docs/README.md b/docs/README.md index b7a24b9..39c218a 100644 --- a/docs/README.md +++ b/docs/README.md @@ -9,14 +9,15 @@ reading source code or every doc. | [ARCHITECTURE.md](ARCHITECTURE.md) | Understand the pipeline end-to-end: data flow, output artifacts, how mutants are injected and evaluated. | | [MODULE_REFERENCE.md](MODULE_REFERENCE.md) | Call or modify an existing function — every public function's signature, inputs, outputs, and side effects, without opening source files. | | [CONFIG.md](CONFIG.md) | Read, write, or validate a `config.yml` — full schema, every key. | -| [EXTENDING.md](EXTENDING.md) | Add a new mutant-generation approach, execution backend, or LLM provider — exact checklists of files to touch. | +| [EXTENDING.md](EXTENDING.md) | Add a new mutant-generation approach, execution backend, language, or LLM provider — exact checklists of files to touch. | | [DEVELOPMENT.md](DEVELOPMENT.md) | Set up, run commands, run tests, follow conventions, or avoid known gotchas and open bugs. | Quick orientation (enough for trivial tasks, no further reading needed): -- *multiplex* mutates **one Java method** per run: extract it with tree-sitter, - ask an LLM to generate mutants, splice each mutant back into the source file, - run the project's tests, and record which mutants survive. +- *multiplex* mutates **one method/function** per run (Java or Python, set by + `project.language`): extract it with tree-sitter, ask an LLM to generate + mutants, splice each mutant back into the source file, run the project's tests, + and record which mutants survive. - Entry point: `multiplex/__main__.py`, run as `uv run ./multiplex ./config.yml`. - Python >= 3.13 (pin 3.13 — see DEVELOPMENT.md), deps via `uv`, lint via `ruff`, tests via `pytest` from the repo root. diff --git a/examples/README.md b/examples/README.md index 19b6713..e13f13b 100644 --- a/examples/README.md +++ b/examples/README.md @@ -1,18 +1,31 @@ -# Example run +# Example runs -A self-contained, end-to-end example of *multiplex*. +Self-contained, end-to-end examples of *multiplex* — one Java, one Python. ```bash # from the repo root -uv run multiplex ./examples/config.yml +uv run ./multiplex ./examples/config-java.yml # Java +uv run ./multiplex ./examples/config-python.yml # Python ``` -## What it does +## What the Java example does Mutates one method — `com.example.Classifier.classify` in the small Maven -project under [`project/example/`](project/example) — using the **basic** -approach, then runs that project's JUnit tests against each mutant with the -**mvn** backend to see which mutants survive. +project under [`project/java-example/`](project/java-example) — using the +**basic** approach, then runs that project's JUnit tests against each mutant with +the **mvn** backend to see which mutants survive. + +## What the Python example does + +Mutates one function — `classify` in +[`project/python-example/classifier.py`](project/python-example/classifier.py) — +using the **basic** approach with `project.language: python`, then runs that +project's pytest suite against each mutant with the **pytest** backend. Output +lands under `project/python-example/output/basic-mutants/` (mutant files are +`.py`; the summary CSV columns are identical to the Java run). + +Prerequisite: pytest for the active interpreter (already provided by `uv run`) +plus an LLM endpoint as below. The rest of this page describes the Java example. ## Prerequisites @@ -20,20 +33,20 @@ approach, then runs that project's JUnit tests against each mutant with the `mvn clean test`; the project targets Java 11). If `mvn` uses an older JDK — e.g. via `JAVA_HOME` — the build fails with "release version 11 not supported"; point `JAVA_HOME` at a JDK 11+ or lower `maven.compiler.release` - in `project/example/pom.xml`. + in `project/java-example/pom.xml`. - A running [Ollama](https://ollama.com) serving the model named in - `config.yml` under `llm.model` (default `gpt-oss:20b`): + `config-java.yml` under `llm.model` (default `gpt-oss:20b`): ```bash ollama pull gpt-oss:20b # or edit llm.model to a model you already have ``` Any LiteLLM-supported provider works — edit the `llm` section of - `config.yml` (`model`, `endpoint`, `token_env_var`). + `config-java.yml` (`model`, `endpoint`, `token_env_var`). ## Output -Written under `project/example/output/basic-mutants/`: +Written under `project/java-example/output/basic-mutants/`: - `mutant_N.java` — each generated mutant (the mutated method body). - `mutant_summary.csv` — one row per mutant with columns @@ -46,6 +59,7 @@ git-ignored. ## Trying another approach Prompt sets are optional per approach, so you can switch `mutation.approach` in -`config.yml` to `hazop` or `stpa` without editing anything else — their prompts -are already included. `mutahunter` and `llmorpheus` need their own prompts added -first (see the comments in `config.yml` and the top-level README). +`config-java.yml` to `hazop` or `stpa` without editing anything else — their +prompts are already included. `mutahunter` and `llmorpheus` need their own +prompts added first (see the comments in `config-java.yml` and the top-level +README). diff --git a/examples/config.yml b/examples/config-java.yml similarity index 93% rename from examples/config.yml rename to examples/config-java.yml index 2e2043d..4fc2ea6 100644 --- a/examples/config.yml +++ b/examples/config-java.yml @@ -1,15 +1,15 @@ -# config.yml — self-contained example. -# Run from the repo root: uv run multiplex ./examples/config.yml +# config-java.yml — self-contained Java example. +# Run from the repo root: uv run ./multiplex ./examples/config-java.yml # # Requirements for this example: # - Maven (`mvn`) and a JDK 11+ on PATH (used by the `mvn` runtool below; -# the example project targets Java 11 — see project/example/pom.xml). +# the example project targets Java 11 — see project/java-example/pom.xml). # - A running Ollama serving the model named under `llm.model` # (`ollama pull gpt-oss:20b`, or point `llm.model` at any model you have). project: - projectroot: examples/project/example - filename: examples/project/example/src/main/java/com/example/Classifier.java + projectroot: examples/project/java-example + filename: examples/project/java-example/src/main/java/com/example/Classifier.java method: classify line: 6 runtool: mvn diff --git a/examples/config-python.yml b/examples/config-python.yml new file mode 100644 index 0000000..03f55c2 --- /dev/null +++ b/examples/config-python.yml @@ -0,0 +1,111 @@ +# config-python.yml — self-contained Python example. +# Run from the repo root: uv run ./multiplex ./examples/config-python.yml +# +# Requirements for this example: +# - pytest available for the active interpreter (it is a dev dependency of +# this repo, so `uv run` already provides it) — used by the `pytest` runtool. +# - A running Ollama serving the model named under `llm.model` +# (`ollama pull gpt-oss:20b`, or point `llm.model` at any model you have). + +project: + language: python + projectroot: examples/project/python-example + filename: examples/project/python-example/classifier.py + method: classify + line: 4 + runtool: pytest + +mutation: + approach: basic + +llm: + model: ollama/gpt-oss:20b + endpoint: http://127.0.0.1:11434 + token_env_var: # env var name storing token (NOT the token itself) + +system_prompts: + # Only the prompts for the selected `mutation.approach` are required. + # `basic` (the approach used above) needs only `basic_generate_mutants`. + basic_generate_mutants: | + You are a code generator. + When given a Python function, re-implement it with a single small behavioral defect (a mutant). + Change exactly one thing: an operator, a boundary condition, a constant, or a returned value. + Include a concise code comment describing the injected defect. + Output the defective Python code only and nothing else. + + # The prompts below let you switch `mutation.approach` to hazop or stpa + # without editing anything else. + hazop_describe_process: | + When given a Python function, describe the intent of the code in a line-by-line manner, as concisely as possible. + + Explain what the code does, not how it is written. + End each line with a comma and don't use commas anywhere else. + Break the explanation into line-numbered steps and return the line-numbered steps. + hazop_identify_deviations: | + For each numbered description, output one version of the string where one of the rules has changed. + The rule must be changed using one of the "guidewords" from HAZOP. + The guidewords and their interpretation is delimited by ###, in the form guideword - interpretation. + That is, everything before - is the guideword and everything after is the meaning. + + ### + + No (not, none) - None of the design intent is achieved + More (more of, higher) - Quantitative increase in a parameter + Less (less of, lower) - Quantitative decrease in a parameter + As well as (more than) - An additional activity occurs + Part of - Only some of the design intention is achieved + Reverse - Logical opposite of the design intent occurs + Other than (other) - Complete substitution (another activity takes place or an unusual activity occurs or uncommon condition exists) + + ### + + Please output each new rule, as a csv line, without headings in the format: + "number, original_rule, guideword (CAPITALIZED), changed_rule" + hazop_implement_deviations: | + You are a code generator. + When given a description, re-implement the function but with the incorrect behavior/defect as described in the description. + + Include a concise code comment to describe the implemented defect. + Output the defective Python code only and nothing else. + stpa_describe_control_flow: | + You describe Python functions as control flow diagrams. + When given a Python function, create a written description in the DOT language used in GraphViz of the control structure. + Use human readable labels and descriptions for each node. + Output the control diagram only, and nothing else. + stpa_identify_ucas: | + Identify 10 different Unsafe Control Actions (UCAs) within the control diagram described in DOT language, referencing the code, using each of the guidewords delimited by ### below: + + ### + * provides + * does not provide + * too early + * too late + * out of order + * stopped too soon + * applied for too long + + ### + + You must describe each UCA in the format: + + An example of this is: + variable x calculated out of order when sorting has not occurred. + + Output the numbered list of UCAs with no empty lines, and nothing else. + stpa_implement_ucas: | + You are a code generator. + When given a description, re-implement the function but with the incorrect behavior/defect as described in the description. + + Include a concise code comment to describe the implemented defect. + Output the defective Python code only and nothing else. + + # llmorpheus fills a in a code fragment with a buggy variant. + llmorpheus_system: | + You are an expert Python programmer applying mutation testing. + You will be given a Python code fragment containing a single PLACEHOLDER. + Replace the PLACEHOLDER with a short code fragment that is syntactically + valid Python and changes the behavior of the original fragment. + Follow the response template you are given exactly. + + # The mutahunter approach needs its own prompt to run: + # - mutahunter: prompts must be supplied by you (licensing) — see README. diff --git a/examples/project/example/pom.xml b/examples/project/java-example/pom.xml similarity index 100% rename from examples/project/example/pom.xml rename to examples/project/java-example/pom.xml diff --git a/examples/project/example/src/main/java/com/example/Classifier.java b/examples/project/java-example/src/main/java/com/example/Classifier.java similarity index 100% rename from examples/project/example/src/main/java/com/example/Classifier.java rename to examples/project/java-example/src/main/java/com/example/Classifier.java diff --git a/examples/project/example/src/test/java/com/example/ClassifierTest.java b/examples/project/java-example/src/test/java/com/example/ClassifierTest.java similarity index 100% rename from examples/project/example/src/test/java/com/example/ClassifierTest.java rename to examples/project/java-example/src/test/java/com/example/ClassifierTest.java diff --git a/examples/project/python-example/classifier.py b/examples/project/python-example/classifier.py new file mode 100644 index 0000000..6c89720 --- /dev/null +++ b/examples/project/python-example/classifier.py @@ -0,0 +1,9 @@ +"""Classifies integers by sign. Target of the multiplex Python example run.""" + + +def classify(n): + if n > 0: + return "positive" + if n < 0: + return "negative" + return "zero" diff --git a/examples/project/python-example/test_classifier.py b/examples/project/python-example/test_classifier.py new file mode 100644 index 0000000..4fa806e --- /dev/null +++ b/examples/project/python-example/test_classifier.py @@ -0,0 +1,13 @@ +from classifier import classify + + +def test_classifies_positive(): + assert classify(5) == "positive" + + +def test_classifies_negative(): + assert classify(-5) == "negative" + + +def test_classifies_zero(): + assert classify(0) == "zero" diff --git a/multiplex/__main__.py b/multiplex/__main__.py index df46142..b7528f3 100644 --- a/multiplex/__main__.py +++ b/multiplex/__main__.py @@ -1,24 +1,24 @@ import argparse import os import shutil -import yaml from pathlib import Path import approach.basic.controller as basic -import approach.mutahunter.controller as mutahunter -import approach.stpa.controller as stpa import approach.hazop.controller as hazop import approach.llmorpheus.controller as llmorpheus -from execute import maven, defects4j - +import approach.mutahunter.controller as mutahunter +import approach.stpa.controller as stpa +import yaml +from execute import defects4j, maven, pytest_runner +from languages import get_language from model import Model from prompts import resolve_prompts -from util.io import reset_source_code from util.extract_method import extract_method_from_file +from util.io import reset_source_code def main(): - s = """ + s = r""" # # # # # # # # # # # # # # # # # # # # # # # # # _ _ _ _ # @@ -41,22 +41,25 @@ def main(): args = parser.parse_args() - config = yaml.safe_load(open(args.config)) + config = None + with open(args.config, "r") as config_file: + config = yaml.safe_load(config_file) - output_path = Path(config['project']['projectroot'], 'output/') - duplicate_file_path = Path(config['project']['filename'] + ".orig") + output_path = Path(config["project"]["projectroot"], "output/") + duplicate_file_path = Path(config["project"]["filename"] + ".orig") - # Validate the approach and its required prompts before any destructive work - # (output wipe, source reset) below. - approach = config['mutation']['approach'] + approach = config["mutation"]["approach"] prompts = resolve_prompts(config, approach) + language = get_language(config["project"].get("language")) print(f"File: {config['project']['filename']}") print(f"Method: {config['project']['method']}") if output_path.exists(): - # response = input(f"Output dir ({output_path}) already exists. Would you like to delete it and continue? (y/n)") - response = "y" + response = input( + f"Output dir ({output_path}) already exists. Would you like to delete it and continue? (y/n)" + ) + # response = "y" if response == "y": shutil.rmtree(output_path) @@ -67,10 +70,14 @@ def main(): else: os.makedirs(output_path, exist_ok=True) - reset_source_code(duplicate_file_path, config['project']['filename']) + reset_source_code(duplicate_file_path, config["project"]["filename"]) method_span = extract_method_from_file( - config['project']['filename'], config['project']['method'], output_path, config['project']['line'] + config["project"]["filename"], + config["project"]["method"], + output_path, + config["project"]["line"], + language, ) if method_span is None: raise SystemExit( @@ -80,35 +87,58 @@ def main(): ) method_start_byte, method_end_byte = method_span - model = Model(model=config['llm']['model'], endpoint=config['llm']['endpoint'], - api_key_var=config['llm']['token_env_var']) + model = Model( + model=config["llm"]["model"], + endpoint=config["llm"]["endpoint"], + api_key_var=config["llm"]["token_env_var"], + ) if approach == "stpa": - stpa.main(model, output_path, prompts) + stpa.main(model, output_path, prompts, language) elif approach == "hazop": - hazop.main(model, output_path, prompts) + hazop.main(model, output_path, prompts, language) elif approach == "basic": - basic.main(model, output_path, prompts) + basic.main(model, output_path, prompts, language) elif approach == "mutahunter": - mutahunter.main(model, output_path, prompts) + mutahunter.main(model, output_path, prompts, language) elif approach == "llmorpheus": - llmorpheus.main(model, output_path, prompts) + llmorpheus.main(model, output_path, prompts, language) - if config['project']['runtool'] == "mvn": + if config["project"]["runtool"] == "mvn": maven.run_mutants( - config['project']['projectroot'], - config['project']['filename'], + config["project"]["projectroot"], + config["project"]["filename"], + output_path, + method_start_byte, + method_end_byte, + duplicate_file_path, + config["mutation"]["approach"], + language, + ) + elif config["project"]["runtool"] == "d4j": + defects4j.run_mutants( + config["project"]["projectroot"], + config["project"]["filename"], + output_path, + method_start_byte, + method_end_byte, + duplicate_file_path, + config["mutation"]["approach"], + language, + ) + elif config["project"]["runtool"] == "pytest": + pytest_runner.run_mutants( + config["project"]["projectroot"], + config["project"]["filename"], output_path, method_start_byte, method_end_byte, duplicate_file_path, - config['mutation']['approach'], + config["mutation"]["approach"], + language, ) - elif config['project']['runtool'] == "d4j": - defects4j.run_mutants(config['project']['projectroot'], config['project']['filename'], output_path, - method_start_byte, method_end_byte, duplicate_file_path, config['mutation']['approach']) - reset_source_code(duplicate_file_path, config['project']['filename']) + reset_source_code(duplicate_file_path, config["project"]["filename"]) if __name__ == "__main__": diff --git a/multiplex/approach/basic/code_generator.py b/multiplex/approach/basic/code_generator.py index e166c50..95ccaea 100644 --- a/multiplex/approach/basic/code_generator.py +++ b/multiplex/approach/basic/code_generator.py @@ -5,10 +5,10 @@ from util.io import read_input_to_str, write_to_file -def _get_system_prompt(method_under_test, system_prompt): +def _get_system_prompt(method_under_test, system_prompt, language): return (system_prompt + f"""\n - The original Java Method is delimited below using ###. + The original {language.noun} is delimited below using ###. ### {method_under_test} @@ -16,22 +16,22 @@ def _get_system_prompt(method_under_test, system_prompt): """) -def generate_code(model, output_dir, system_prompt): +def generate_code(model, output_dir, system_prompt, language): """Generate mutated versions of the method.""" - method_under_test_file_path = Path(output_dir, "original_method.java") + method_under_test_file_path = language.original_method_path(output_dir) method_under_test = read_input_to_str(method_under_test_file_path) mutants_dir = Path(output_dir, "basic-mutants/") os.makedirs(mutants_dir, exist_ok=True) messages = [ - {"content": _get_system_prompt(method_under_test, system_prompt), "role": "system"} + {"content": _get_system_prompt(method_under_test, system_prompt, language), "role": "system"} ] for count in range(0, 10): mutant = model.make_request(messages) - mutant = mutant.removeprefix("```java") + mutant = mutant.removeprefix("```" + language.fence) mutant = mutant.split("```")[0] os.makedirs(mutants_dir, exist_ok=True) - mutant_file_path = Path(mutants_dir, f"mutant_{str(count)}.java") + mutant_file_path = Path(mutants_dir, f"mutant_{str(count)}{language.extension}") write_to_file(mutant_file_path, mutant) diff --git a/multiplex/approach/basic/controller.py b/multiplex/approach/basic/controller.py index 24444c3..cc7ae00 100644 --- a/multiplex/approach/basic/controller.py +++ b/multiplex/approach/basic/controller.py @@ -2,11 +2,11 @@ from approach.basic.code_generator import generate_code -def main(model, output_dir, prompts): +def main(model, output_dir, prompts, language): """Controller for Basic prompt approach""" - generate_code(model, output_dir, prompts['basic_generate_mutants']) + generate_code(model, output_dir, prompts['basic_generate_mutants'], language) -def __main__(model, output_dir, prompts): - main(model, output_dir, prompts) +def __main__(model, output_dir, prompts, language): + main(model, output_dir, prompts, language) diff --git a/multiplex/approach/hazop/code_generator.py b/multiplex/approach/hazop/code_generator.py index 842347a..b9996c5 100644 --- a/multiplex/approach/hazop/code_generator.py +++ b/multiplex/approach/hazop/code_generator.py @@ -6,30 +6,30 @@ from util.io import read_hazop_mutations, read_input_to_str, write_to_file -def _get_system_prompt(method_under_test, system_prompt): +def _get_system_prompt(method_under_test, system_prompt, language): return (system_prompt + f"""\n - The original Java Method is delimited below using ###. - + The original {language.noun} is delimited below using ###. + ### {method_under_test} ### """) -def generate_code(model, output_dir, system_prompt): +def generate_code(model, output_dir, system_prompt, language): """Generate code for each mutant.""" mutate_descriptions_output_path = Path(output_dir, "hazop-mutated-descriptions.txt") mutated_descriptions, count = read_hazop_mutations(mutate_descriptions_output_path) - original_method_path = Path(output_dir, "original_method.java") + original_method_path = language.original_method_path(output_dir) original_method = read_input_to_str(original_method_path) mutants_dir = Path(output_dir, "hazop-mutants/") messages_orig = [ - {"content": _get_system_prompt(original_method, system_prompt), "role": "system"}, + {"content": _get_system_prompt(original_method, system_prompt, language), "role": "system"}, ] count = 0 @@ -39,10 +39,10 @@ def generate_code(model, output_dir, system_prompt): messages.append({"content": user_prompt, "role": "user"}) mutant = model.make_request(messages) - mutant = mutant.removeprefix("```java") + mutant = mutant.removeprefix("```" + language.fence) mutant = mutant.split("```", 1)[0] os.makedirs(mutants_dir, exist_ok=True) - mutant_file_path = Path(mutants_dir, f"mutant_{str(count)}.java") + mutant_file_path = Path(mutants_dir, f"mutant_{str(count)}{language.extension}") write_to_file(mutant_file_path, mutant) count += 1 diff --git a/multiplex/approach/hazop/controller.py b/multiplex/approach/hazop/controller.py index 6e60e83..b469c16 100644 --- a/multiplex/approach/hazop/controller.py +++ b/multiplex/approach/hazop/controller.py @@ -6,13 +6,13 @@ from approach.hazop.code_generator import generate_code -def main(model, output_dir, prompts): +def main(model, output_dir, prompts, language): """Controller for HAZOP""" - describe_method(model, output_dir, prompts['hazop_describe_process']) + describe_method(model, output_dir, prompts['hazop_describe_process'], language) mutate_descriptions(model, output_dir, prompts['hazop_identify_deviations']) - generate_code(model, output_dir, prompts['hazop_implement_deviations']) + generate_code(model, output_dir, prompts['hazop_implement_deviations'], language) -def __main__(model, output_dir, prompts): - main(model, output_dir, prompts) +def __main__(model, output_dir, prompts, language): + main(model, output_dir, prompts, language) diff --git a/multiplex/approach/hazop/method_explainer.py b/multiplex/approach/hazop/method_explainer.py index 94f41a4..2f50b8f 100644 --- a/multiplex/approach/hazop/method_explainer.py +++ b/multiplex/approach/hazop/method_explainer.py @@ -6,9 +6,9 @@ from util.io import write_to_file -def describe_method(model, output_dir, system_prompt): +def describe_method(model, output_dir, system_prompt, language): """Describe the method.""" - user_prompt = get_method_under_test(output_dir) + user_prompt = get_method_under_test(output_dir, language) messages = [ {"content": system_prompt, "role": "system"}, diff --git a/multiplex/approach/llmorpheus/code_generator.py b/multiplex/approach/llmorpheus/code_generator.py index 322f2dc..d6af28c 100644 --- a/multiplex/approach/llmorpheus/code_generator.py +++ b/multiplex/approach/llmorpheus/code_generator.py @@ -46,7 +46,7 @@ def _user_prompt(code: str, orig: str): """ -def generate_code(model: Model, output_dir: Path, llmorpheus_system: str): +def generate_code(model: Model, output_dir: Path, llmorpheus_system: str, language): placeholder_dir = Path(output_dir, "placeholders") mutants_dir = Path(output_dir, "llmorpheus-mutants/") mutants_dir.mkdir(parents=True, exist_ok=True) @@ -57,13 +57,13 @@ def generate_code(model: Model, output_dir: Path, llmorpheus_system: str): if not isinstance(locations, dict): raise ValueError("Expected a JSON object at the top level.") - placeholders_count = len(list(placeholder_dir.glob("*_placeholder.java"))) + placeholders_count = len(list(placeholder_dir.glob(f"*_placeholder{language.extension}"))) mutant_number = 0 for count in range(0, placeholders_count): - p_f = f"{count}_placeholder.java" + p_f = f"{count}_placeholder{language.extension}" p_fn = Path(placeholder_dir, p_f) - orig_p_fn = Path(placeholder_dir, f"{count}_orig.java") + orig_p_fn = Path(placeholder_dir, f"{count}_orig{language.extension}") with open(p_fn, "r") as pf: p = pf.read() @@ -81,11 +81,11 @@ def generate_code(model: Model, output_dir: Path, llmorpheus_system: str): for mutant_code in matches: - mutant_file_path = Path(mutants_dir, f"mutant_{str(mutant_number)}.java") + mutant_file_path = Path(mutants_dir, f"mutant_{str(mutant_number)}{language.extension}") start_byte = locations[p_f]["start_byte"] end_byte = locations[p_f]["end_byte"] - method_under_test = get_method_under_test(output_dir) + method_under_test = get_method_under_test(output_dir, language) code_bytes = method_under_test.encode() mutant = ( code_bytes[:start_byte] + mutant_code.encode() + code_bytes[end_byte:] diff --git a/multiplex/approach/llmorpheus/controller.py b/multiplex/approach/llmorpheus/controller.py index 00cd9f1..3a8a89b 100644 --- a/multiplex/approach/llmorpheus/controller.py +++ b/multiplex/approach/llmorpheus/controller.py @@ -7,12 +7,12 @@ from model import Model -def main(model: Model, output_dir: Path, prompts: dict[str, Any]): +def main(model: Model, output_dir: Path, prompts: dict[str, Any], language): """Controller for LLMorpheus prompt approach""" - create_placeholders(output_dir) - generate_code(model, output_dir, prompts['llmorpheus_system']) - + create_placeholders(output_dir, language) + generate_code(model, output_dir, prompts['llmorpheus_system'], language) -def __main__(model: Model, output_dir: Path, prompts: dict[str, Any]): - main(model, output_dir, prompts) + +def __main__(model: Model, output_dir: Path, prompts: dict[str, Any], language): + main(model, output_dir, prompts, language) diff --git a/multiplex/approach/llmorpheus/placeholders.py b/multiplex/approach/llmorpheus/placeholders.py index 16bbc6c..6b52ddb 100644 --- a/multiplex/approach/llmorpheus/placeholders.py +++ b/multiplex/approach/llmorpheus/placeholders.py @@ -1,12 +1,12 @@ +import json from pathlib import Path -import tree_sitter_java as ts_java -from tree_sitter import Language, Node, Parser + +from tree_sitter import Node, Parser + from approach.util import get_method_under_test -import json PLACEHOLDER = "" -JAVA_LANGUAGE = Language(ts_java.language()) -parser = Parser(JAVA_LANGUAGE) + JAVA_QUERY = """ ; --- (i) IF, SWITCH, WHILE, DO-WHILE --- (if_statement condition: (parenthesized_expression (_) @condition.if)) @@ -35,15 +35,47 @@ (method_invocation name: (identifier) @call.name object: (_)? @call.receiver - arguments: (argument_list + arguments: (argument_list (_) @call.arg ) @call.arg_sequence_content ) """ +# Python has no parenthesized_expression wrapper around conditions, and uses +# for/left/right (iterator protocol) rather than C-style for headers. Operators +# are their own node types, which stand in for Java's operator-bearing sites. +PYTHON_QUERY = """ +; --- (i) IF / WHILE CONDITIONS --- +(if_statement condition: (_) @condition.if) +(while_statement condition: (_) @condition.while) + +; --- (ii) FOR LOOPS --- +(for_statement left: (_) @loop.target right: (_) @loop.iterable) + +; --- (iii) OPERATOR EXPRESSIONS --- +(comparison_operator) @expr.comparison +(boolean_operator) @expr.boolean +(binary_operator) @expr.binary + +; --- (iv) FUNCTION CALLS --- +(call + function: (_) @call.function + arguments: (argument_list + (_) @call.arg + ) +) +""" + +# Keyed by language name. Adding a language means adding its mutation-site query +# here (see docs/EXTENDING.md § Add a language). +MUTATION_QUERIES = { + "java": JAVA_QUERY, + "python": PYTHON_QUERY, +} + -def find_placeholder_locations(root, source_code): - query = JAVA_LANGUAGE.query(JAVA_QUERY) +def find_placeholder_locations(root, source_code, language): + query = language.ts_language.query(MUTATION_QUERIES[language.name]) captures: dict[str, list[Node]] = query.captures(root) results = [] @@ -71,15 +103,16 @@ def add_placeholder(m, code_bytes): ) -def create_placeholders(output_dir: Path): - method_under_test = get_method_under_test(output_dir) +def create_placeholders(output_dir: Path, language): + method_under_test = get_method_under_test(output_dir, language) code_bytes = method_under_test.encode() + parser = Parser(language.ts_language) tree = parser.parse(code_bytes) root_node = tree.root_node - matches = find_placeholder_locations(root_node, method_under_test) + matches = find_placeholder_locations(root_node, method_under_test, language) count = 0 placeholder_dir = Path(output_dir, "placeholders") @@ -89,12 +122,12 @@ def create_placeholders(output_dir: Path): for m in matches: p = add_placeholder(m, code_bytes) orig_p = m["text"] - - p_f = f"{count}_placeholder.java" + + p_f = f"{count}_placeholder{language.extension}" p_fn = Path(placeholder_dir, p_f) locations_dict[p_f] = m - orig_p_fn = Path(placeholder_dir, f"{count}_orig.java") + orig_p_fn = Path(placeholder_dir, f"{count}_orig{language.extension}") p_fn.write_text(p if isinstance(p, str) else p.decode()) orig_p_fn.write_text(orig_p if isinstance(orig_p, str) else orig_p.decode()) diff --git a/multiplex/approach/mutahunter/code_generator.py b/multiplex/approach/mutahunter/code_generator.py index 76ccba1..e88ab19 100644 --- a/multiplex/approach/mutahunter/code_generator.py +++ b/multiplex/approach/mutahunter/code_generator.py @@ -2,18 +2,17 @@ import os from pathlib import Path -from tree_sitter import Language, Parser -import tree_sitter_java as ts_java +from tree_sitter import Parser from util.parser import add_mutant_to_method, parse_output from util.io import read_input_to_str, write_to_file -def _get_system_prompt(method_under_test, system_prompt): +def _get_system_prompt(method_under_test, system_prompt, language): return ( system_prompt + f"""\n - The original Java Method is delimited below using ###. + The original {language.noun} is delimited below using ###. ### {method_under_test} @@ -45,11 +44,10 @@ def _get_user_prompt(ast, src_code_file, language, numbered_src_code): return user_prompt -def _get_ast(method_under_test): +def _get_ast(method_under_test, language): """Get ast of method.""" bytestring = bytes(method_under_test.encode("utf-8")) - language = Language(ts_java.language()) - parser = Parser(language) + parser = Parser(language.ts_language) tree = parser.parse(bytestring, encoding="utf8") return str(tree.root_node) @@ -62,26 +60,25 @@ def _get_numbered_src_code(method_under_test): return numbered_src_code -def generate_code(model, output_dir, system_prompt): +def generate_code(model, output_dir, system_prompt, language): """Generate mutated versions of the method.""" - method_under_test_file_path = Path(output_dir, "original_method.java") + method_under_test_file_path = language.original_method_path(output_dir) method_under_test = read_input_to_str(method_under_test_file_path) mutants_dir = Path(output_dir, "mutahunter-mutants/") os.makedirs(mutants_dir, exist_ok=True) - ast = _get_ast(method_under_test) + ast = _get_ast(method_under_test, language) src_code_file = method_under_test_file_path - language = "Java" numbered_src_code = _get_numbered_src_code(method_under_test) messages = [ { - "content": _get_system_prompt(method_under_test, system_prompt), + "content": _get_system_prompt(method_under_test, system_prompt, language), "role": "system", }, { "content": _get_user_prompt( - ast, src_code_file, language, numbered_src_code + ast, src_code_file, language.label, numbered_src_code ), "role": "user", }, @@ -98,5 +95,5 @@ def generate_code(model, output_dir, system_prompt): mutant_yaml["line_number"], ) - mutant_file_path = Path(mutants_dir, f"mutant_{str(count)}.java") + mutant_file_path = Path(mutants_dir, f"mutant_{str(count)}{language.extension}") write_to_file(mutant_file_path, mutant) diff --git a/multiplex/approach/mutahunter/controller.py b/multiplex/approach/mutahunter/controller.py index eb474e1..0cc33d3 100644 --- a/multiplex/approach/mutahunter/controller.py +++ b/multiplex/approach/mutahunter/controller.py @@ -2,11 +2,11 @@ from approach.mutahunter.code_generator import generate_code -def main(model, output_dir, prompts): +def main(model, output_dir, prompts, language): """Controller for Mutahunter prompt approach""" - generate_code(model, output_dir, prompts['mutahunter_generate_mutants']) + generate_code(model, output_dir, prompts['mutahunter_generate_mutants'], language) -def __main__(model, output_dir, prompts): - main(model, output_dir, prompts) +def __main__(model, output_dir, prompts, language): + main(model, output_dir, prompts, language) diff --git a/multiplex/approach/stpa/code_generator.py b/multiplex/approach/stpa/code_generator.py index aa9f6a2..f05a323 100644 --- a/multiplex/approach/stpa/code_generator.py +++ b/multiplex/approach/stpa/code_generator.py @@ -6,10 +6,10 @@ from util.io import read_input_to_str, read_ucas, write_to_file -def _get_system_prompt(method_under_test, system_prompt): +def _get_system_prompt(method_under_test, system_prompt, language): return (system_prompt + f"""\n - The original Java Method is delimited below using ###. + The original {language.noun} is delimited below using ###. ### {method_under_test} @@ -17,17 +17,17 @@ def _get_system_prompt(method_under_test, system_prompt): """) -def generate_code(model, output_dir, system_prompt): +def generate_code(model, output_dir, system_prompt, language): """Generate mutants from UCAs""" uca_file_path = Path(output_dir, "ucas.csv") - method_under_test_file_path = Path(output_dir, "original_method.java") + method_under_test_file_path = language.original_method_path(output_dir) ucas, ucas_count = read_ucas(uca_file_path) method_under_test = read_input_to_str(method_under_test_file_path) mutants_dir = Path(output_dir, "stpa-mutants/") messages_orig = [ - {"content": _get_system_prompt(method_under_test, system_prompt), "role": "system"} + {"content": _get_system_prompt(method_under_test, system_prompt, language), "role": "system"} ] for count in range(0, ucas_count): @@ -36,9 +36,9 @@ def generate_code(model, output_dir, system_prompt): messages.append({"content": user_prompt, "role": "user"}) mutant = model.make_request(messages) - mutant = mutant.removeprefix("```java") + mutant = mutant.removeprefix("```" + language.fence) mutant = mutant.split("```", 1)[0] os.makedirs(mutants_dir, exist_ok=True) - mutant_file_path = Path(mutants_dir, f"mutant_{str(count)}.java") + mutant_file_path = Path(mutants_dir, f"mutant_{str(count)}{language.extension}") write_to_file(mutant_file_path, mutant) diff --git a/multiplex/approach/stpa/control_diagram.py b/multiplex/approach/stpa/control_diagram.py index c259d75..84257bb 100644 --- a/multiplex/approach/stpa/control_diagram.py +++ b/multiplex/approach/stpa/control_diagram.py @@ -6,9 +6,9 @@ from util.io import write_to_file -def create_control_diagram(model, output_dir, system_prompt): +def create_control_diagram(model, output_dir, system_prompt, language): """Create control description from method""" - user_prompt = get_method_under_test(output_dir) + user_prompt = get_method_under_test(output_dir, language) messages = [ {"content": system_prompt, "role": "system"}, {"content": user_prompt, "role": "user"}, diff --git a/multiplex/approach/stpa/controller.py b/multiplex/approach/stpa/controller.py index 02ec7c3..7704971 100644 --- a/multiplex/approach/stpa/controller.py +++ b/multiplex/approach/stpa/controller.py @@ -5,13 +5,13 @@ from approach.stpa.uca_generator import generate_ucas -def main(model, output_dir, prompts): +def main(model, output_dir, prompts, language): """Controller for Systems Theoretic Process Analysis (STPA)""" - create_control_diagram(model, output_dir, prompts['stpa_describe_control_flow']) + create_control_diagram(model, output_dir, prompts['stpa_describe_control_flow'], language) generate_ucas(model, output_dir, prompts['stpa_identify_ucas']) - generate_code(model, output_dir, prompts['stpa_implement_ucas']) + generate_code(model, output_dir, prompts['stpa_implement_ucas'], language) -def __main__(model, output_dir, prompts): - main(model, output_dir, prompts) +def __main__(model, output_dir, prompts, language): + main(model, output_dir, prompts, language) diff --git a/multiplex/approach/util.py b/multiplex/approach/util.py index 35f2f7a..909cdb5 100644 --- a/multiplex/approach/util.py +++ b/multiplex/approach/util.py @@ -1,9 +1,8 @@ -from pathlib import Path from util.io import read_input_to_str -def get_method_under_test(output_dir): - method_under_test_path = Path(output_dir, "original_method.java") +def get_method_under_test(output_dir, language): + method_under_test_path = language.original_method_path(output_dir) method_under_test = read_input_to_str(method_under_test_path) if method_under_test is not None: return method_under_test diff --git a/multiplex/checks/compilable.py b/multiplex/checks/compilable.py index c161099..eeea2dc 100644 --- a/multiplex/checks/compilable.py +++ b/multiplex/checks/compilable.py @@ -1,5 +1,4 @@ -from tree_sitter import Language, Node, Parser -import tree_sitter_java as ts_java +from tree_sitter import Node, Parser def tree_has_for_errors(node: Node): @@ -19,10 +18,9 @@ def _read_file(fn): return bytes(src.encode("utf-8")) -def check_mutant_compilable(filename): - """Check if Java file contains compilation errors""" - language = Language(ts_java.language()) - parser = Parser(language) +def check_mutant_compilable(filename, language): + """Check if the source file contains parse (syntax) errors.""" + parser = Parser(language.ts_language) tree = parser.parse(_read_file(filename), encoding="utf8") return not tree_has_for_errors(tree.root_node) diff --git a/multiplex/checks/syntactic_equivalence.py b/multiplex/checks/syntactic_equivalence.py index 24717f0..9409c8b 100644 --- a/multiplex/checks/syntactic_equivalence.py +++ b/multiplex/checks/syntactic_equivalence.py @@ -1,18 +1,17 @@ """Check if mutants are syntactically equivalent to original code""" from difflib import SequenceMatcher -from tree_sitter import Language, Parser -import tree_sitter_java as ts_java +from tree_sitter import Parser -def _get_clean_tree(root_node) -> str: +def _get_clean_tree(root_node, comment_node_types) -> str: """Serialize a parse tree to a string for comparison. - Comments and whitespace are ignored. + Comments and whitespace are ignored. Leaf tokens include their source text to include values in comparison. """ def traverse(node) -> str: - if node.type in ("comment", "line_comment", "block_comment", "whitespace"): + if node.type in comment_node_types or node.type == "whitespace": return "" if node.child_count == 0: return node.type + "-" + node.text.decode("utf-8") @@ -21,10 +20,10 @@ def traverse(node) -> str: return traverse(root_node) -def _tree_is_equivalent(original_root_node, mutant_root_node): +def _tree_is_equivalent(original_root_node, mutant_root_node, comment_node_types): """Check tree for equivalence.""" - mutant = _get_clean_tree(mutant_root_node) - original = _get_clean_tree(original_root_node) + mutant = _get_clean_tree(mutant_root_node, comment_node_types) + original = _get_clean_tree(original_root_node, comment_node_types) seq_match = SequenceMatcher(None, original, mutant) ratio = seq_match.ratio() @@ -41,10 +40,15 @@ def _read_file(fn): return bytes(src.encode("utf-8")) -def check_mutant_equivalent(mutant_filename, original_filename) -> bool: - """Check if mutant file tree is equal to original tree.""" - language = Language(ts_java.language()) - parser = Parser(language) +def check_mutant_equivalent(mutant_filename, original_filename, language) -> bool: + """Check if mutant file tree is equal to original tree. + + ``language`` is a ``languages.LanguageSpec`` selecting the tree-sitter + grammar and which node types count as comments (ignored in the comparison). + """ + parser = Parser(language.ts_language) mutant_tree = parser.parse(_read_file(mutant_filename), encoding="utf-8") original_tree = parser.parse(_read_file(original_filename), encoding="utf-8") - return _tree_is_equivalent(original_tree.root_node, mutant_tree.root_node) + return _tree_is_equivalent( + original_tree.root_node, mutant_tree.root_node, language.comment_node_types + ) diff --git a/multiplex/execute/defects4j.py b/multiplex/execute/defects4j.py index d83f652..8fa2db3 100644 --- a/multiplex/execute/defects4j.py +++ b/multiplex/execute/defects4j.py @@ -65,11 +65,12 @@ def run_mutants( method_end_byte, duplicate, approach, + language, ): """Execute all mutants using Defects4J build.""" mutants_dir = Path(output_path, approach + "-mutants") mutant_files = [f for f in os.listdir(mutants_dir) if isfile(join(mutants_dir, f))] - original_method_path = Path(output_path, "original_method.java") + original_method_path = language.original_method_path(output_path) mutants = [] header = ["MUTANT", "EQUIVALENCE", "COMPILABLE", "SURVIVES"] @@ -91,12 +92,12 @@ def run_mutants( shutil.copy2(duplicate, original_file) path = Path(mutants_dir, mutant_file) - mutant_equivalent = check_mutant_equivalent(path, original_method_path) + mutant_equivalent = check_mutant_equivalent(path, original_method_path, language) mutant_output.append(mutant_equivalent) rewrite_method(original_file, method_start_byte, method_end_byte, path) - mutant_compiles = check_mutant_compilable(original_file) + mutant_compiles = check_mutant_compilable(original_file, language) mutant_output.append(mutant_compiles) mutant_survives = False diff --git a/multiplex/execute/maven.py b/multiplex/execute/maven.py index 1ed2714..f2a06f0 100644 --- a/multiplex/execute/maven.py +++ b/multiplex/execute/maven.py @@ -40,11 +40,12 @@ def run_mutants( method_end_byte, duplicate, approach, + language, ): """Execute all mutants using a Maven build.""" mutants_dir = Path(output_path, approach + "-mutants") mutant_files = [f for f in os.listdir(mutants_dir) if isfile(join(mutants_dir, f))] - original_method_path = Path(output_path, "original_method.java") + original_method_path = language.original_method_path(output_path) mutants = [["MUTANT", "EQUIVALENCE", "COMPILABLE", "SURVIVES"]] @@ -62,12 +63,12 @@ def run_mutants( shutil.copy2(duplicate, original_file) path = Path(mutants_dir, mutant_file) - mutant_equivalent = check_mutant_equivalent(path, original_method_path) + mutant_equivalent = check_mutant_equivalent(path, original_method_path, language) mutant_output.append(mutant_equivalent) rewrite_method(original_file, method_start_byte, method_end_byte, path) - mutant_compiles = check_mutant_compilable(original_file) + mutant_compiles = check_mutant_compilable(original_file, language) mutant_output.append(mutant_compiles) mutant_survives = False diff --git a/multiplex/execute/pytest_runner.py b/multiplex/execute/pytest_runner.py new file mode 100644 index 0000000..e4dfc41 --- /dev/null +++ b/multiplex/execute/pytest_runner.py @@ -0,0 +1,91 @@ +"""Pytest execution backend. + +Mirrors ``execute/maven.py``: baseline the original (unmutated) project, then per +mutant restore the source, run the equivalence/compilable checks, splice the +mutant into the source, and run the project's tests to decide survived/killed. + +Named ``pytest_runner`` (not ``pytest``) so it does not shadow the installed +``pytest`` package on import. +""" + +import os +import shutil +import subprocess +import sys +from os.path import isfile, join +from pathlib import Path + +from checks.compilable import check_mutant_compilable +from checks.syntactic_equivalence import check_mutant_equivalent +from util.io import write_mutant_summary +from util.rewrite_method import rewrite_method + + +def _execute(project_root): + """Run the project's tests with pytest. + + Returns True if pytest exits 0 (all tests pass) — i.e. the mutant is not + detected and survives — and False otherwise (mutant killed, incl. collection + or syntax errors pytest reports). + """ + command = [sys.executable, "-m", "pytest", "-q", str(project_root)] + try: + result = subprocess.run( + command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, check=False + ) + except FileNotFoundError as exc: + raise SystemExit( + "pytest is not installed for the current interpreter " + f"({sys.executable}). Install pytest to use the 'pytest' runtool." + ) from exc + + return result.returncode == 0 + + +def run_mutants( + project_root, + original_file, + output_path, + method_start_byte, + method_end_byte, + duplicate, + approach, + language, +): + """Execute all mutants using pytest.""" + mutants_dir = Path(output_path, approach + "-mutants") + mutant_files = [f for f in os.listdir(mutants_dir) if isfile(join(mutants_dir, f))] + original_method_path = language.original_method_path(output_path) + + mutants = [["MUTANT", "EQUIVALENCE", "COMPILABLE", "SURVIVES"]] + + if not _execute(project_root): + raise IOError( + "The original (unmutated) project did not pass pytest, so mutants " + "cannot be evaluated against it. Run it manually to see why: " + f"{sys.executable} -m pytest {project_root}" + ) + + for mutant_file in mutant_files: + mutant_output = [mutant_file] + print("Evaluating Mutant: ", mutant_file) + if duplicate.exists(): + shutil.copy2(duplicate, original_file) + + path = Path(mutants_dir, mutant_file) + mutant_equivalent = check_mutant_equivalent(path, original_method_path, language) + mutant_output.append(mutant_equivalent) + + rewrite_method(original_file, method_start_byte, method_end_byte, path) + + mutant_compiles = check_mutant_compilable(original_file, language) + mutant_output.append(mutant_compiles) + + mutant_survives = False + if mutant_compiles: + mutant_survives = _execute(project_root) + mutant_output.append(str(mutant_survives)) + + mutants.append(mutant_output) + + write_mutant_summary(mutants_dir, mutants) diff --git a/multiplex/languages/__init__.py b/multiplex/languages/__init__.py new file mode 100644 index 0000000..f23572b --- /dev/null +++ b/multiplex/languages/__init__.py @@ -0,0 +1,80 @@ +"""Per-language facts for the mutation pipeline. + +A ``LanguageSpec`` carries everything the language-agnostic pipeline needs to +work with a given source language: the tree-sitter grammar, the node types that +denote a mutable definition, which nodes count as comments (ignored by the +syntactic-equivalence check), the code-fence tag and prompt wording the +approaches use, and the on-disk extension for the method/mutant artifacts. + +Adding a language means adding one ``LanguageSpec`` to ``_REGISTRY`` (plus a +prompt set and an example) — see docs/EXTENDING.md § Add a language. +""" + +from dataclasses import dataclass +from pathlib import Path + +from tree_sitter import Language +import tree_sitter_java as ts_java +import tree_sitter_python as ts_python + +# Base name (without extension) of the method-under-test artifact every approach +# reads from the output directory. +_METHOD_STEM = "original_method" + + +@dataclass(frozen=True) +class LanguageSpec: + """Everything the pipeline needs to know about one source language.""" + + name: str + extension: str + ts_language: Language + def_node_types: frozenset + comment_node_types: frozenset + fence: str + noun: str + label: str + + def original_method_path(self, output_dir) -> Path: + """Path of the extracted method-under-test artifact for this language.""" + return Path(output_dir, _METHOD_STEM + self.extension) + + +_REGISTRY = { + "java": LanguageSpec( + name="java", + extension=".java", + ts_language=Language(ts_java.language()), + def_node_types=frozenset({"method_declaration", "constructor_declaration"}), + comment_node_types=frozenset({"comment", "line_comment", "block_comment"}), + fence="java", + noun="Java method", + label="Java", + ), + "python": LanguageSpec( + name="python", + extension=".py", + ts_language=Language(ts_python.language()), + def_node_types=frozenset({"function_definition"}), + comment_node_types=frozenset({"comment"}), + fence="python", + noun="Python function", + label="Python", + ), +} + +DEFAULT_LANGUAGE = "java" + + +def get_language(name=None) -> LanguageSpec: + """Return the ``LanguageSpec`` for ``name`` (defaults to Java). + + Raises ``SystemExit`` with an actionable message if ``name`` is unknown, so + a bad ``project.language`` fails fast before any destructive step — mirroring + ``prompts.resolve_prompts``. + """ + key = (name or DEFAULT_LANGUAGE).lower() + if key not in _REGISTRY: + valid = ", ".join(sorted(_REGISTRY)) + raise SystemExit(f"Invalid language '{name}'. Choose one of: {valid}") + return _REGISTRY[key] diff --git a/multiplex/util/extract_method.py b/multiplex/util/extract_method.py index 7c45eb3..48b321c 100644 --- a/multiplex/util/extract_method.py +++ b/multiplex/util/extract_method.py @@ -1,17 +1,13 @@ import logging -from pathlib import Path -import tree_sitter_java as ts_java -from tree_sitter import Language, Parser -JAVA_LANGUAGE = Language(ts_java.language()) -OUTPUT_FILE = "original_method.java" +from tree_sitter import Parser -def extract_method_from_file(file_path, method_name, output_dir, start_line): +def extract_method_from_file(file_path, method_name, output_dir, start_line, language): """Extract method from source code.""" - output_file = Path(output_dir, OUTPUT_FILE) + output_file = language.original_method_path(output_dir) - parser = Parser(JAVA_LANGUAGE) + parser = Parser(language.ts_language) with open(file_path, "r") as f: code = f.read() @@ -22,7 +18,7 @@ def extract_method_from_file(file_path, method_name, output_dir, start_line): def find_method(node): - if node.type == "method_declaration" or node.type == "constructor_declaration": + if node.type in language.def_node_types: for child in node.children: if ( child.type == "identifier" diff --git a/pyproject.toml b/pyproject.toml index df5ac9f..52dc08a 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -5,11 +5,12 @@ description = "⚠️ A Modular LLM-based Mutation Tool." readme = "README.md" requires-python = ">=3.13" dependencies = [ - "litellm>=1.76.3", + "litellm>=1.84.1", "openai>=1.104.2", "pyyaml>=6.0.2", "tree-sitter==0.23.2", "tree-sitter-java==0.23.5", + "tree-sitter-python>=0.23,<0.24", ] [dependency-groups] diff --git a/tests/approach/__init__.py b/tests/approach/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/tests/approach/conftest.py b/tests/approach/conftest.py new file mode 100644 index 0000000..9963c71 --- /dev/null +++ b/tests/approach/conftest.py @@ -0,0 +1,15 @@ +"""Test setup for the mutant-generation approaches. + +Approach modules (like the execution backends) use flat imports — `from util.io +import ...`, `from approach.basic.code_generator import ...` — because at runtime +the tool runs as a directory with `multiplex/` on `sys.path`. They cannot be +imported as `multiplex.approach.`, so their tests import them the runtime way. +Add `multiplex/` to `sys.path` here. (Mirrors tests/execute/conftest.py.) +""" + +import sys +from pathlib import Path + +MULTIPLEX_DIR = Path(__file__).resolve().parents[2] / "multiplex" +if str(MULTIPLEX_DIR) not in sys.path: + sys.path.insert(0, str(MULTIPLEX_DIR)) diff --git a/tests/approach/test_basic_language.py b/tests/approach/test_basic_language.py new file mode 100644 index 0000000..7c95466 --- /dev/null +++ b/tests/approach/test_basic_language.py @@ -0,0 +1,70 @@ +"""The `basic` approach must honor the run's language. + +`basic` is the single-prompt template every other approach follows, so these +lock in the language-aware behavior shared across all five: the mutant file +extension, the code-fence that gets stripped, and the prompt noun. The LLM is a +stub (no network) — the docs' "keep pure logic separate from make_request" +convention makes this testable. +""" + +from approach.basic.code_generator import generate_code +from languages import get_language + +PYTHON = get_language("python") +JAVA = get_language("java") + + +class FakeModel: + """Records the messages it is handed and returns a fixed fenced response.""" + + def __init__(self, response): + self.response = response + self.calls = [] + + def make_request(self, messages): + self.calls.append(messages) + return self.response + + +def _seed_method(tmp_path, language, source): + language.original_method_path(tmp_path).write_text(source) + + +def test_python_run_writes_py_mutants_and_strips_python_fence(tmp_path): + _seed_method(tmp_path, PYTHON, "def f(n):\n return n\n") + model = FakeModel("```python\ndef f(n):\n return n + 1\n```") + + generate_code(model, tmp_path, "SYSTEM", PYTHON) + + mutants = sorted((tmp_path / "basic-mutants").glob("*")) + assert len(mutants) == 10 # basic asks 10 times + assert all(m.suffix == ".py" for m in mutants) + # the ```python opening fence and closing ``` are both removed + body = mutants[0].read_text() + assert "```" not in body + assert body.strip() == "def f(n):\n return n + 1" + + +def test_java_run_writes_java_mutants_and_strips_java_fence(tmp_path): + _seed_method(tmp_path, JAVA, "int f(int n){ return n; }") + model = FakeModel("```java\nint f(int n){ return n + 1; }\n```") + + generate_code(model, tmp_path, "SYSTEM", JAVA) + + mutants = sorted((tmp_path / "basic-mutants").glob("*")) + assert mutants # non-empty + assert all(m.suffix == ".java" for m in mutants) + body = mutants[0].read_text() + assert "```" not in body + assert body.strip() == "int f(int n){ return n + 1; }" + + +def test_system_prompt_uses_language_noun(tmp_path): + _seed_method(tmp_path, PYTHON, "def f(n):\n return n\n") + model = FakeModel("```python\ndef f(n):\n return n\n```") + + generate_code(model, tmp_path, "BASE PROMPT", PYTHON) + + system_content = model.calls[0][0]["content"] + assert "Python function" in system_content # language.noun + assert "Java" not in system_content diff --git a/tests/checks/test_compiles.py b/tests/checks/test_compiles.py index 2ccec1f..9b474f2 100644 --- a/tests/checks/test_compiles.py +++ b/tests/checks/test_compiles.py @@ -2,6 +2,10 @@ from pathlib import Path from multiplex.checks.compilable import check_mutant_compilable +from multiplex.languages import get_language + +JAVA = get_language("java") +PYTHON = get_language("python") @pytest.fixture @@ -16,8 +20,31 @@ def uncompilable_mutant_path(): def test_check_mutant_compiles(original_method_path): - assert check_mutant_compilable(original_method_path) + assert check_mutant_compilable(original_method_path, JAVA) def test_check_mutant_does_not_compile(uncompilable_mutant_path): - assert not check_mutant_compilable(uncompilable_mutant_path) + assert not check_mutant_compilable(uncompilable_mutant_path, JAVA) + + +# --- Python --- + + +def test_python_valid_source_compiles(tmp_path): + p = tmp_path / "m.py" + p.write_text("def f(n):\n return n + 1\n") + assert check_mutant_compilable(p, PYTHON) + + +@pytest.mark.parametrize( + "code", + [ + "def f(n):\n return foo(n", # unclosed paren + "def f(n)\n return n", # missing colon + "def f(n):\n return n +* 2", # bad operator + ], +) +def test_python_syntax_error_does_not_compile(tmp_path, code): + p = tmp_path / "m.py" + p.write_text(code) + assert not check_mutant_compilable(p, PYTHON) diff --git a/tests/checks/test_syntactic_equivalence.py b/tests/checks/test_syntactic_equivalence.py index 0b3b4a2..7212839 100644 --- a/tests/checks/test_syntactic_equivalence.py +++ b/tests/checks/test_syntactic_equivalence.py @@ -2,6 +2,10 @@ from pathlib import Path from multiplex.checks.syntactic_equivalence import check_mutant_equivalent +from multiplex.languages import get_language + +JAVA = get_language("java") +PYTHON = get_language("python") @pytest.fixture @@ -19,12 +23,12 @@ def killed_mutant_path(): def test_check_mutant_equivalent_is_equivalent(equivalent_mutant_path, original_method_path): """Check mutant identified as equivalent.""" - assert check_mutant_equivalent(equivalent_mutant_path, original_method_path) + assert check_mutant_equivalent(equivalent_mutant_path, original_method_path, JAVA) def test_check_mutant_equivalent_not_equivalent(killed_mutant_path, original_method_path): """Check mutant identified as not equivalent.""" - assert not check_mutant_equivalent(killed_mutant_path, original_method_path) + assert not check_mutant_equivalent(killed_mutant_path, original_method_path, JAVA) def _write(tmp_path, name, code): @@ -37,29 +41,56 @@ def test_string_literal_change_is_not_equivalent(tmp_path): """A changed string literal (same AST shape) must not be equivalent.""" original = _write(tmp_path, "o.java", 'String f() { return "negative"; }') mutant = _write(tmp_path, "m.java", 'String f() { return "positive"; }') - assert not check_mutant_equivalent(mutant, original) + assert not check_mutant_equivalent(mutant, original, JAVA) def test_numeric_literal_change_is_not_equivalent(tmp_path): original = _write(tmp_path, "o.java", "int f() { return 1; }") mutant = _write(tmp_path, "m.java", "int f() { return 2; }") - assert not check_mutant_equivalent(mutant, original) + assert not check_mutant_equivalent(mutant, original, JAVA) def test_identifier_change_is_not_equivalent(tmp_path): original = _write(tmp_path, "o.java", "int f(int a, int b) { return a; }") mutant = _write(tmp_path, "m.java", "int f(int a, int b) { return b; }") - assert not check_mutant_equivalent(mutant, original) + assert not check_mutant_equivalent(mutant, original, JAVA) def test_comment_only_change_is_equivalent(tmp_path): """Differences only in comments/whitespace remain equivalent.""" original = _write(tmp_path, "o.java", "int f() { return 1; }") mutant = _write(tmp_path, "m.java", "int f() {\n // added comment\n return 1;\n}") - assert check_mutant_equivalent(mutant, original) + assert check_mutant_equivalent(mutant, original, JAVA) def test_identical_source_is_equivalent(tmp_path): original = _write(tmp_path, "o.java", "int f() { return 1; }") mutant = _write(tmp_path, "m.java", "int f() { return 1; }") - assert check_mutant_equivalent(mutant, original) + assert check_mutant_equivalent(mutant, original, JAVA) + + +# --- Python --- + + +def test_python_numeric_change_is_not_equivalent(tmp_path): + original = _write(tmp_path, "o.py", "def f():\n return 1\n") + mutant = _write(tmp_path, "m.py", "def f():\n return 2\n") + assert not check_mutant_equivalent(mutant, original, PYTHON) + + +def test_python_operator_change_is_not_equivalent(tmp_path): + original = _write(tmp_path, "o.py", "def f(a, b):\n return a + b\n") + mutant = _write(tmp_path, "m.py", "def f(a, b):\n return a - b\n") + assert not check_mutant_equivalent(mutant, original, PYTHON) + + +def test_python_comment_only_change_is_equivalent(tmp_path): + original = _write(tmp_path, "o.py", "def f():\n return 1\n") + mutant = _write(tmp_path, "m.py", "def f():\n # injected defect\n return 1\n") + assert check_mutant_equivalent(mutant, original, PYTHON) + + +def test_python_identical_source_is_equivalent(tmp_path): + original = _write(tmp_path, "o.py", "def f(n):\n return n * 2\n") + mutant = _write(tmp_path, "m.py", "def f(n):\n return n * 2\n") + assert check_mutant_equivalent(mutant, original, PYTHON) diff --git a/tests/execute/test_defects4j.py b/tests/execute/test_defects4j.py index e77e94c..cc774cc 100644 --- a/tests/execute/test_defects4j.py +++ b/tests/execute/test_defects4j.py @@ -8,6 +8,9 @@ # `execute.defects4j` is imported the runtime way (multiplex/ on sys.path); see # tests/execute/conftest.py. from execute import defects4j +from languages import get_language + +JAVA = get_language("java") METHOD = "int f(int n) { return n; }" @@ -134,6 +137,7 @@ def _run(project, approach="basic"): project["end"], project["duplicate"], approach, + JAVA, ) diff --git a/tests/execute/test_maven.py b/tests/execute/test_maven.py index 31eb1b3..1a6c768 100644 --- a/tests/execute/test_maven.py +++ b/tests/execute/test_maven.py @@ -7,6 +7,9 @@ # `execute.maven` is imported the runtime way (multiplex/ on sys.path); see # tests/execute/conftest.py. from execute import maven +from languages import get_language + +JAVA = get_language("java") METHOD = "int f(int n) { return n; }" @@ -118,6 +121,7 @@ def _run(project, approach="basic"): project["end"], project["duplicate"], approach, + JAVA, ) diff --git a/tests/execute/test_pytest_runner.py b/tests/execute/test_pytest_runner.py new file mode 100644 index 0000000..f2fddc0 --- /dev/null +++ b/tests/execute/test_pytest_runner.py @@ -0,0 +1,168 @@ +import csv +import sys +import types +from pathlib import Path + +import pytest + +# `execute.pytest_runner` is imported the runtime way (multiplex/ on sys.path); +# see tests/execute/conftest.py. +from execute import pytest_runner +from languages import get_language + +PYTHON = get_language("python") + +METHOD = "def f(n):\n return n\n" + + +# --------------------------------------------------------------------------- # +# _execute: maps a pytest run to a survives/killed boolean # +# --------------------------------------------------------------------------- # + + +def _run_returning(returncode): + def run(command, *args, **kwargs): + return types.SimpleNamespace(returncode=returncode) + + return run + + +def test_execute_true_on_zero_exit(monkeypatch): + monkeypatch.setattr(pytest_runner.subprocess, "run", _run_returning(0)) + assert pytest_runner._execute("proj") is True + + +def test_execute_false_on_nonzero_exit(monkeypatch): + monkeypatch.setattr(pytest_runner.subprocess, "run", _run_returning(1)) + assert pytest_runner._execute("proj") is False + + +def test_execute_uses_argv_list_not_shell(monkeypatch): + captured = {} + + def fake_run(command, *args, **kwargs): + captured["command"] = command + captured["kwargs"] = kwargs + return types.SimpleNamespace(returncode=0) + + monkeypatch.setattr(pytest_runner.subprocess, "run", fake_run) + + assert pytest_runner._execute("/path with spaces/proj") is True + assert isinstance(captured["command"], list) + # Runs pytest under multiplex's own interpreter, not a PATH-resolved `python`. + assert captured["command"][:3] == [sys.executable, "-m", "pytest"] + assert "/path with spaces/proj" in captured["command"] + assert captured["kwargs"].get("shell", False) is False + + +def test_execute_reports_missing_pytest_clearly(monkeypatch): + def raise_fnf(*a, **k): + raise FileNotFoundError(2, "No such file or directory", sys.executable) + + monkeypatch.setattr(pytest_runner.subprocess, "run", raise_fnf) + with pytest.raises(SystemExit) as exc: + pytest_runner._execute("proj") + assert "pytest" in str(exc.value).lower() + + +# --------------------------------------------------------------------------- # +# run_mutants: full evaluation loop over a mutants directory # +# --------------------------------------------------------------------------- # + + +@pytest.fixture +def project(tmp_path): + """A synthetic Python source file + backup, output dir, and three mutants.""" + source = METHOD + src = tmp_path / "mod.py" + src.write_text(source) + (tmp_path / "mod.py.orig").write_text(source) + + output = tmp_path / "output" + output.mkdir() + (output / "original_method.py").write_text(METHOD) + + mutants = output / "basic-mutants" + mutants.mkdir() + # equivalent: only a comment differs -> AST-equal, compiles, survives + (mutants / "mutant_equivalent.py").write_text("# same behaviour\n" + METHOD) + # killed: changes behaviour -> compiles, tests fail + (mutants / "mutant_killed.py").write_text("def f(n):\n return n + 1\n") + # broken: unterminated call -> does not parse, tests never run + (mutants / "mutant_broken.py").write_text("def f(n):\n return foo(n\n") + + return { + "project_root": str(tmp_path), + "src": str(src), + "duplicate": Path(str(src) + ".orig"), + "output": output, + "mutants": mutants, + "start": source.index(METHOD), + "end": source.index(METHOD) + len(METHOD), + } + + +def _run(project, approach="basic"): + pytest_runner.run_mutants( + project["project_root"], + project["src"], + project["output"], + project["start"], + project["end"], + project["duplicate"], + approach, + PYTHON, + ) + + +def _summary(project): + with open(Path(project["mutants"], "mutant_summary.csv")) as f: + rows = list(csv.reader(f)) + return rows[0], {r[0]: r[1:] for r in rows[1:]} + + +def test_run_mutants_writes_summary_with_correct_classification(project, monkeypatch): + src = project["src"] + + # Stand in for `pytest`: the suite fails (mutant killed) only when the + # injected source returns n + 1; every other state passes. + monkeypatch.setattr( + pytest_runner, "_execute", lambda project_root: "return n + 1" not in Path(src).read_text() + ) + + _run(project) + + header, data = _summary(project) + assert header == ["MUTANT", "EQUIVALENCE", "COMPILABLE", "SURVIVES"] + assert len(data) == 3 + # [EQUIVALENCE, COMPILABLE, SURVIVES] + assert data["mutant_equivalent.py"] == ["True", "True", "True"] + assert data["mutant_killed.py"] == ["False", "True", "False"] + assert data["mutant_broken.py"][1:] == ["False", "False"] # not compilable, killed + + +def test_run_mutants_raises_when_baseline_fails(project, monkeypatch): + monkeypatch.setattr(pytest_runner, "_execute", lambda project_root: False) + with pytest.raises(IOError): + _run(project) + + +def test_run_mutants_skips_tests_for_noncompilable_mutant(project, monkeypatch): + # Keep only the non-compilable mutant. + for f in project["mutants"].glob("*.py"): + f.unlink() + (project["mutants"] / "mutant_broken.py").write_text("def f(n):\n return foo(n\n") + + calls = {"n": 0} + + def counting_execute(project_root): + calls["n"] += 1 + return True + + monkeypatch.setattr(pytest_runner, "_execute", counting_execute) + _run(project) + + # Only the baseline runs the suite; the non-compilable mutant is not tested. + assert calls["n"] == 1 + _, data = _summary(project) + assert data["mutant_broken.py"][1:] == ["False", "False"] diff --git a/tests/test_languages.py b/tests/test_languages.py new file mode 100644 index 0000000..e4bc6c1 --- /dev/null +++ b/tests/test_languages.py @@ -0,0 +1,46 @@ +from pathlib import Path + +import pytest + +from multiplex.languages import DEFAULT_LANGUAGE, get_language + + +def test_get_python_spec_fields(): + py = get_language("python") + assert py.name == "python" + assert py.extension == ".py" + assert "function_definition" in py.def_node_types + assert "comment" in py.comment_node_types + assert py.fence == "python" + assert py.label == "Python" + + +def test_get_java_spec_fields(): + java = get_language("java") + assert java.name == "java" + assert java.extension == ".java" + assert "method_declaration" in java.def_node_types + + +def test_default_is_java(): + assert DEFAULT_LANGUAGE == "java" + assert get_language(None).name == "java" + assert get_language().name == "java" + + +def test_original_method_path_uses_extension(): + assert get_language("python").original_method_path("out") == Path("out/original_method.py") + assert get_language("java").original_method_path("out") == Path("out/original_method.java") + + +def test_unknown_language_raises_systemexit(): + with pytest.raises(SystemExit) as exc: + get_language("ruby") + msg = str(exc.value) + assert "ruby" in msg + assert "java" in msg and "python" in msg + + +def test_language_name_is_case_insensitive(): + assert get_language("Python").name == "python" + assert get_language("JAVA").name == "java" diff --git a/tests/util/test_extract_method.py b/tests/util/test_extract_method.py index 7fa5b5b..1ea791a 100644 --- a/tests/util/test_extract_method.py +++ b/tests/util/test_extract_method.py @@ -2,6 +2,10 @@ import tempfile import pytest # noqa: F401 from multiplex.util.extract_method import extract_method_from_file +from multiplex.languages import get_language + +JAVA = get_language("java") +PYTHON = get_language("python") # --- Tests constructed using OpenCode Assistant with GPT-4.1 --- @@ -24,7 +28,7 @@ def test_extracts_constructor_by_name_and_line(java_constructor_code): with open(src_path, "w") as f: f.write(code) # Constructor name is 'Example', line is 2 (1-based) - result = extract_method_from_file(src_path, "Example", tmpdir, start_line=3) + result = extract_method_from_file(src_path, "Example", tmpdir, 3, JAVA) assert result is not None, "Should extract constructor declaration" start, end = result with open(os.path.join(tmpdir, "original_method.java")) as f: @@ -40,7 +44,7 @@ def test_does_not_extract_nonexistent_constructor(java_constructor_code): with open(src_path, "w") as f: f.write(code) # Wrong name - result = extract_method_from_file(src_path, "Nonexistent", tmpdir, start_line=2) + result = extract_method_from_file(src_path, "Nonexistent", tmpdir, 2, JAVA) assert result is None, "Should not extract if constructor name does not match" @@ -51,7 +55,7 @@ def test_extracts_method_not_constructor(java_constructor_code): with open(src_path, "w") as f: f.write(code) # Method name is 'foo', line is 5 (1-based) - result = extract_method_from_file(src_path, "foo", tmpdir, start_line=6) + result = extract_method_from_file(src_path, "foo", tmpdir, 6, JAVA) assert result is not None, "Should extract method declaration" start, end = result with open(os.path.join(tmpdir, "original_method.java")) as f: @@ -59,3 +63,43 @@ def test_extracts_method_not_constructor(java_constructor_code): assert "public void foo()" in extracted # --- End generated tests --- + + +# --- Python extraction --- + +PYTHON_CODE = '''\ +def helper(x): + return x + 1 + + +def classify(n): + if n > 0: + return 1 + return 0 +''' + + +def test_extracts_python_function_by_name_and_line(): + with tempfile.TemporaryDirectory() as tmpdir: + src_path = os.path.join(tmpdir, "mod.py") + with open(src_path, "w") as f: + f.write(PYTHON_CODE) + # 'classify' identifier is on line 5 (1-based) + result = extract_method_from_file(src_path, "classify", tmpdir, 5, PYTHON) + assert result is not None, "Should extract Python function" + with open(os.path.join(tmpdir, "original_method.py")) as f: + extracted = f.read() + assert extracted.startswith("def classify(n):") + assert "return 1" in extracted + # must not spill into the following/preceding function + assert "helper" not in extracted + + +def test_python_line_must_match(): + with tempfile.TemporaryDirectory() as tmpdir: + src_path = os.path.join(tmpdir, "mod.py") + with open(src_path, "w") as f: + f.write(PYTHON_CODE) + # correct name, wrong line + result = extract_method_from_file(src_path, "classify", tmpdir, 1, PYTHON) + assert result is None diff --git a/uv.lock b/uv.lock index c48c487..466268a 100644 --- a/uv.lock +++ b/uv.lock @@ -178,14 +178,32 @@ wheels = [ [[package]] name = "fastuuid" -version = "0.12.0" +version = "0.14.0" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/19/17/13146a1e916bd2971d0a58db5e0a4ad23efdd49f78f33ac871c161f8007b/fastuuid-0.12.0.tar.gz", hash = "sha256:d0bd4e5b35aad2826403f4411937c89e7c88857b1513fe10f696544c03e9bd8e", size = 19180, upload-time = "2025-01-27T18:04:14.387Z" } +sdist = { url = "https://files.pythonhosted.org/packages/c3/7d/d9daedf0f2ebcacd20d599928f8913e9d2aea1d56d2d355a93bfa2b611d7/fastuuid-0.14.0.tar.gz", hash = "sha256:178947fc2f995b38497a74172adee64fdeb8b7ec18f2a5934d037641ba265d26", size = 18232, upload-time = "2025-10-19T22:19:22.402Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/a9/e8/d2bb4f19e5ee15f6f8e3192a54a897678314151aa17d0fb766d2c2cbc03d/fastuuid-0.12.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:7fe2407316a04ee8f06d3dbc7eae396d0a86591d92bafe2ca32fce23b1145786", size = 247512, upload-time = "2025-01-27T18:04:08.115Z" }, - { url = "https://files.pythonhosted.org/packages/bc/53/25e811d92fd60f5c65e098c3b68bd8f1a35e4abb6b77a153025115b680de/fastuuid-0.12.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b9b31dd488d0778c36f8279b306dc92a42f16904cba54acca71e107d65b60b0c", size = 258257, upload-time = "2025-01-27T18:03:56.408Z" }, - { url = "https://files.pythonhosted.org/packages/10/23/73618e7793ea0b619caae2accd9e93e60da38dd78dd425002d319152ef2f/fastuuid-0.12.0-cp313-cp313-manylinux_2_34_x86_64.whl", hash = "sha256:b19361ee649365eefc717ec08005972d3d1eb9ee39908022d98e3bfa9da59e37", size = 278559, upload-time = "2025-01-27T18:03:58.661Z" }, - { url = "https://files.pythonhosted.org/packages/e4/41/6317ecfc4757d5f2a604e5d3993f353ba7aee85fa75ad8b86fce6fc2fa40/fastuuid-0.12.0-cp313-cp313-win_amd64.whl", hash = "sha256:8fc66b11423e6f3e1937385f655bedd67aebe56a3dcec0cb835351cfe7d358c9", size = 157276, upload-time = "2025-01-27T18:06:39.245Z" }, + { url = "https://files.pythonhosted.org/packages/a5/83/ae12dd39b9a39b55d7f90abb8971f1a5f3c321fd72d5aa83f90dc67fe9ed/fastuuid-0.14.0-cp313-cp313-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl", hash = "sha256:77a09cb7427e7af74c594e409f7731a0cf887221de2f698e1ca0ebf0f3139021", size = 510720, upload-time = "2025-10-19T22:42:34.633Z" }, + { url = "https://files.pythonhosted.org/packages/53/b0/a4b03ff5d00f563cc7546b933c28cb3f2a07344b2aec5834e874f7d44143/fastuuid-0.14.0-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:9bd57289daf7b153bfa3e8013446aa144ce5e8c825e9e366d455155ede5ea2dc", size = 262024, upload-time = "2025-10-19T22:30:25.482Z" }, + { url = "https://files.pythonhosted.org/packages/9c/6d/64aee0a0f6a58eeabadd582e55d0d7d70258ffdd01d093b30c53d668303b/fastuuid-0.14.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:ac60fc860cdf3c3f327374db87ab8e064c86566ca8c49d2e30df15eda1b0c2d5", size = 251679, upload-time = "2025-10-19T22:36:14.096Z" }, + { url = "https://files.pythonhosted.org/packages/60/f5/a7e9cda8369e4f7919d36552db9b2ae21db7915083bc6336f1b0082c8b2e/fastuuid-0.14.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ab32f74bd56565b186f036e33129da77db8be09178cd2f5206a5d4035fb2a23f", size = 277862, upload-time = "2025-10-19T22:36:23.302Z" }, + { url = "https://files.pythonhosted.org/packages/f0/d3/8ce11827c783affffd5bd4d6378b28eb6cc6d2ddf41474006b8d62e7448e/fastuuid-0.14.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:33e678459cf4addaedd9936bbb038e35b3f6b2061330fd8f2f6a1d80414c0f87", size = 278278, upload-time = "2025-10-19T22:29:43.809Z" }, + { url = "https://files.pythonhosted.org/packages/a2/51/680fb6352d0bbade04036da46264a8001f74b7484e2fd1f4da9e3db1c666/fastuuid-0.14.0-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:1e3cc56742f76cd25ecb98e4b82a25f978ccffba02e4bdce8aba857b6d85d87b", size = 301788, upload-time = "2025-10-19T22:36:06.825Z" }, + { url = "https://files.pythonhosted.org/packages/fa/7c/2014b5785bd8ebdab04ec857635ebd84d5ee4950186a577db9eff0fb8ff6/fastuuid-0.14.0-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:cb9a030f609194b679e1660f7e32733b7a0f332d519c5d5a6a0a580991290022", size = 459819, upload-time = "2025-10-19T22:35:31.623Z" }, + { url = "https://files.pythonhosted.org/packages/01/d2/524d4ceeba9160e7a9bc2ea3e8f4ccf1ad78f3bde34090ca0c51f09a5e91/fastuuid-0.14.0-cp313-cp313-musllinux_1_1_i686.whl", hash = "sha256:09098762aad4f8da3a888eb9ae01c84430c907a297b97166b8abc07b640f2995", size = 478546, upload-time = "2025-10-19T22:26:03.023Z" }, + { url = "https://files.pythonhosted.org/packages/bc/17/354d04951ce114bf4afc78e27a18cfbd6ee319ab1829c2d5fb5e94063ac6/fastuuid-0.14.0-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:1383fff584fa249b16329a059c68ad45d030d5a4b70fb7c73a08d98fd53bcdab", size = 450921, upload-time = "2025-10-19T22:31:02.151Z" }, + { url = "https://files.pythonhosted.org/packages/fb/be/d7be8670151d16d88f15bb121c5b66cdb5ea6a0c2a362d0dcf30276ade53/fastuuid-0.14.0-cp313-cp313-win32.whl", hash = "sha256:a0809f8cc5731c066c909047f9a314d5f536c871a7a22e815cc4967c110ac9ad", size = 154559, upload-time = "2025-10-19T22:36:36.011Z" }, + { url = "https://files.pythonhosted.org/packages/22/1d/5573ef3624ceb7abf4a46073d3554e37191c868abc3aecd5289a72f9810a/fastuuid-0.14.0-cp313-cp313-win_amd64.whl", hash = "sha256:0df14e92e7ad3276327631c9e7cec09e32572ce82089c55cb1bb8df71cf394ed", size = 156539, upload-time = "2025-10-19T22:33:35.898Z" }, + { url = "https://files.pythonhosted.org/packages/16/c9/8c7660d1fe3862e3f8acabd9be7fc9ad71eb270f1c65cce9a2b7a31329ab/fastuuid-0.14.0-cp314-cp314-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl", hash = "sha256:b852a870a61cfc26c884af205d502881a2e59cc07076b60ab4a951cc0c94d1ad", size = 510600, upload-time = "2025-10-19T22:43:44.17Z" }, + { url = "https://files.pythonhosted.org/packages/4c/f4/a989c82f9a90d0ad995aa957b3e572ebef163c5299823b4027986f133dfb/fastuuid-0.14.0-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:c7502d6f54cd08024c3ea9b3514e2d6f190feb2f46e6dbcd3747882264bb5f7b", size = 262069, upload-time = "2025-10-19T22:43:38.38Z" }, + { url = "https://files.pythonhosted.org/packages/da/6c/a1a24f73574ac995482b1326cf7ab41301af0fabaa3e37eeb6b3df00e6e2/fastuuid-0.14.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:1ca61b592120cf314cfd66e662a5b54a578c5a15b26305e1b8b618a6f22df714", size = 251543, upload-time = "2025-10-19T22:32:22.537Z" }, + { url = "https://files.pythonhosted.org/packages/1a/20/2a9b59185ba7a6c7b37808431477c2d739fcbdabbf63e00243e37bd6bf49/fastuuid-0.14.0-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:aa75b6657ec129d0abded3bec745e6f7ab642e6dba3a5272a68247e85f5f316f", size = 277798, upload-time = "2025-10-19T22:33:53.821Z" }, + { url = "https://files.pythonhosted.org/packages/ef/33/4105ca574f6ded0af6a797d39add041bcfb468a1255fbbe82fcb6f592da2/fastuuid-0.14.0-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a8a0dfea3972200f72d4c7df02c8ac70bad1bb4c58d7e0ec1e6f341679073a7f", size = 278283, upload-time = "2025-10-19T22:29:02.812Z" }, + { url = "https://files.pythonhosted.org/packages/fe/8c/fca59f8e21c4deb013f574eae05723737ddb1d2937ce87cb2a5d20992dc3/fastuuid-0.14.0-cp314-cp314-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:1bf539a7a95f35b419f9ad105d5a8a35036df35fdafae48fb2fd2e5f318f0d75", size = 301627, upload-time = "2025-10-19T22:35:54.985Z" }, + { url = "https://files.pythonhosted.org/packages/cb/e2/f78c271b909c034d429218f2798ca4e89eeda7983f4257d7865976ddbb6c/fastuuid-0.14.0-cp314-cp314-musllinux_1_1_aarch64.whl", hash = "sha256:9a133bf9cc78fdbd1179cb58a59ad0100aa32d8675508150f3658814aeefeaa4", size = 459778, upload-time = "2025-10-19T22:28:00.999Z" }, + { url = "https://files.pythonhosted.org/packages/1e/f0/5ff209d865897667a2ff3e7a572267a9ced8f7313919f6d6043aed8b1caa/fastuuid-0.14.0-cp314-cp314-musllinux_1_1_i686.whl", hash = "sha256:f54d5b36c56a2d5e1a31e73b950b28a0d83eb0c37b91d10408875a5a29494bad", size = 478605, upload-time = "2025-10-19T22:36:21.764Z" }, + { url = "https://files.pythonhosted.org/packages/e0/c8/2ce1c78f983a2c4987ea865d9516dbdfb141a120fd3abb977ae6f02ba7ca/fastuuid-0.14.0-cp314-cp314-musllinux_1_1_x86_64.whl", hash = "sha256:ec27778c6ca3393ef662e2762dba8af13f4ec1aaa32d08d77f71f2a70ae9feb8", size = 450837, upload-time = "2025-10-19T22:34:37.178Z" }, + { url = "https://files.pythonhosted.org/packages/df/60/dad662ec9a33b4a5fe44f60699258da64172c39bd041da2994422cdc40fe/fastuuid-0.14.0-cp314-cp314-win32.whl", hash = "sha256:e23fc6a83f112de4be0cc1990e5b127c27663ae43f866353166f87df58e73d06", size = 154532, upload-time = "2025-10-19T22:35:18.217Z" }, + { url = "https://files.pythonhosted.org/packages/1f/f6/da4db31001e854025ffd26bc9ba0740a9cbba2c3259695f7c5834908b336/fastuuid-0.14.0-cp314-cp314-win_amd64.whl", hash = "sha256:df61342889d0f5e7a32f7284e55ef95103f2110fee433c2ae7c2c0956d76ac8a", size = 156457, upload-time = "2025-10-19T22:33:44.579Z" }, ] [[package]] @@ -258,41 +276,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/04/4b/29cac41a4d98d144bf5f6d33995617b185d14b22401f75ca86f384e87ff1/h11-0.16.0-py3-none-any.whl", hash = "sha256:63cf8bbe7522de3bf65932fda1d9c2772064ffb3dae62d55932da54b31cb6c86", size = 37515, upload-time = "2025-04-24T03:35:24.344Z" }, ] -[[package]] -name = "multiplex" -version = "0.1.0" -source = { virtual = "." } -dependencies = [ - { name = "litellm" }, - { name = "openai" }, - { name = "pyyaml" }, - { name = "tree-sitter" }, - { name = "tree-sitter-java" }, -] - -[package.dev-dependencies] -dev = [ - { name = "pylint" }, - { name = "pytest" }, - { name = "ruff" }, -] - -[package.metadata] -requires-dist = [ - { name = "litellm", specifier = ">=1.76.3" }, - { name = "openai", specifier = ">=1.104.2" }, - { name = "pyyaml", specifier = ">=6.0.2" }, - { name = "tree-sitter", specifier = "==0.23.2" }, - { name = "tree-sitter-java", specifier = "==0.23.5" }, -] - -[package.metadata.requires-dev] -dev = [ - { name = "pylint", specifier = ">=3.3.7" }, - { name = "pytest", specifier = ">=8.4.1" }, - { name = "ruff", specifier = ">=0.11.12" }, -] - [[package]] name = "hf-xet" version = "1.1.9" @@ -471,7 +454,7 @@ wheels = [ [[package]] name = "litellm" -version = "1.76.3" +version = "1.93.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "aiohttp" }, @@ -487,9 +470,12 @@ dependencies = [ { name = "tiktoken" }, { name = "tokenizers" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/72/46/57b6539365616452bb6f4401487448ce62e62755738fce55d8222d7a557e/litellm-1.76.3.tar.gz", hash = "sha256:fc81219c59b17b26cc81276ce32582f3715612877ab11c1ea2c26e4853ac67e8", size = 10210403, upload-time = "2025-09-07T01:59:19.55Z" } +sdist = { url = "https://files.pythonhosted.org/packages/93/e1/4f05ca4cbb4efb739c9e66a182ecd5c816bc05bf3665ec8e0fb4ab408379/litellm-1.93.0.tar.gz", hash = "sha256:140bf215e264c71601bca9c06d2436c5451bb59e1e195ea23fc2d3d87b6929ec", size = 15948866, upload-time = "2026-07-19T03:01:24.389Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/d0/d9/5f8ed27241b487f51f04573b8ba06d4460ebed9f792ff5cc148649fbf862/litellm-1.76.3-py3-none-any.whl", hash = "sha256:d62e3ff2a80ec5e551c6d7a0fe199ffe718ecb6cbaa43fc9250dd8d7c0944352", size = 9000797, upload-time = "2025-09-07T01:59:16.261Z" }, + { url = "https://files.pythonhosted.org/packages/f5/e1/eabe3f13d9c8b853a01377b59e78a295f34c24c36b09e5fc1c60c0167f19/litellm-1.93.0-cp313-cp313-manylinux_2_28_aarch64.whl", hash = "sha256:23b8eea4fb8b3b6ade05e7b6085ec9ca12daffcfb38d033d0bbce9c1d5894da5", size = 20164863, upload-time = "2026-07-19T03:01:12.035Z" }, + { url = "https://files.pythonhosted.org/packages/64/49/2db5757f7e284eb12618b547cb62dab49687ffaf1d749ca048210e3d0dbd/litellm-1.93.0-cp313-cp313-manylinux_2_28_x86_64.whl", hash = "sha256:98c15e84d32e922a821c105308bf9ddae8700de9ed396530bee2cbb1cacf4cbb", size = 20157288, upload-time = "2026-07-19T03:01:15.395Z" }, + { url = "https://files.pythonhosted.org/packages/0d/7f/b48d88cb32055b4ba7e51cd67e4ea13a589d4a568d33c3d0dc6994d13b83/litellm-1.93.0-cp314-cp314-manylinux_2_28_aarch64.whl", hash = "sha256:1a476ebc340c070c982eab15b4673fa63a70f5936935f9f20e4d2acb5f35d23b", size = 20165502, upload-time = "2026-07-19T03:01:18.425Z" }, + { url = "https://files.pythonhosted.org/packages/45/6c/65a7f916326daa151131f1fce5e254d4834127a95d53a18f1c4d238dd5c3/litellm-1.93.0-cp314-cp314-manylinux_2_28_x86_64.whl", hash = "sha256:cd70ccd4ba3ef1a395535c287bce72d30c7d937ecca4290c0db37a00738f7ada", size = 20159007, upload-time = "2026-07-19T03:01:21.704Z" }, ] [[package]] @@ -574,9 +560,46 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/fd/69/b547032297c7e63ba2af494edba695d781af8a0c6e89e4d06cf848b21d80/multidict-6.6.4-py3-none-any.whl", hash = "sha256:27d8f8e125c07cb954e54d75d04905a9bba8a439c1d84aca94949d4d03d8601c", size = 12313, upload-time = "2025-08-11T12:08:46.891Z" }, ] +[[package]] +name = "multiplex" +version = "0.1.0" +source = { virtual = "." } +dependencies = [ + { name = "litellm" }, + { name = "openai" }, + { name = "pyyaml" }, + { name = "tree-sitter" }, + { name = "tree-sitter-java" }, + { name = "tree-sitter-python" }, +] + +[package.dev-dependencies] +dev = [ + { name = "pylint" }, + { name = "pytest" }, + { name = "ruff" }, +] + +[package.metadata] +requires-dist = [ + { name = "litellm", specifier = ">=1.84.1" }, + { name = "openai", specifier = ">=1.104.2" }, + { name = "pyyaml", specifier = ">=6.0.2" }, + { name = "tree-sitter", specifier = "==0.23.2" }, + { name = "tree-sitter-java", specifier = "==0.23.5" }, + { name = "tree-sitter-python", specifier = ">=0.23,<0.24" }, +] + +[package.metadata.requires-dev] +dev = [ + { name = "pylint", specifier = ">=3.3.7" }, + { name = "pytest", specifier = ">=8.4.1" }, + { name = "ruff", specifier = ">=0.11.12" }, +] + [[package]] name = "openai" -version = "1.104.2" +version = "2.47.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "anyio" }, @@ -588,9 +611,9 @@ dependencies = [ { name = "tqdm" }, { name = "typing-extensions" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/94/dc/965b3528ed0435b717acca45e2541d94bd827c0520ce172366323c9edcab/openai-1.104.2.tar.gz", hash = "sha256:9b582ead9dd208753f89dae8e36b6548c6ada076e87ba3db36630e29239661ab", size = 557160, upload-time = "2025-09-02T21:42:31.054Z" } +sdist = { url = "https://files.pythonhosted.org/packages/bf/61/9aeef14de759306e85175126d3d6d56ee4f5072a9512c6c171d58d02a62d/openai-2.47.0.tar.gz", hash = "sha256:4e205548acd4304f235b86202269912e55bc88270b15d2a051fa2b53b90343a6", size = 1089906, upload-time = "2026-07-22T17:47:29.723Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/46/9c/d0b56971e5584aea338bb00d3ca96a7f6694dff77006581b21cd773497ce/openai-1.104.2-py3-none-any.whl", hash = "sha256:0148951da12ea651f890ef38f8adef75b78c053dba37ea2bdba857c8945860d4", size = 928160, upload-time = "2025-09-02T21:42:28.678Z" }, + { url = "https://files.pythonhosted.org/packages/41/69/26b032059273ad798d18fbcdbe369e871181841fd8bcb5caee32b7510039/openai-2.47.0-py3-none-any.whl", hash = "sha256:b3a1a7ad974092427ccb46d89f8852bdb67866680bcabeecc3ff5a3fdd71b15b", size = 1639987, upload-time = "2026-07-22T17:47:27.873Z" }, ] [[package]] @@ -1032,6 +1055,21 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/72/57/5bab54d23179350356515526fff3cc0f3ac23bfbc1a1d518a15978d4880e/tree_sitter_java-0.23.5-cp39-abi3-win_arm64.whl", hash = "sha256:402efe136104c5603b429dc26c7e75ae14faaca54cfd319ecc41c8f2534750f4", size = 59059, upload-time = "2024-12-21T18:24:24.934Z" }, ] +[[package]] +name = "tree-sitter-python" +version = "0.23.6" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/1c/30/6766433b31be476fda6569a3a374c2220e45ffee0bff75460038a57bf23b/tree_sitter_python-0.23.6.tar.gz", hash = "sha256:354bfa0a2f9217431764a631516f85173e9711af2c13dbd796a8815acfe505d9", size = 155868, upload-time = "2024-12-22T23:09:55.918Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/ab/67/577a02acae5f776007c924ca86ef14c19c12e71de0aa9d2a036f3c248e7b/tree_sitter_python-0.23.6-cp39-abi3-macosx_10_9_x86_64.whl", hash = "sha256:28fbec8f74eeb2b30292d97715e60fac9ccf8a8091ce19b9d93e9b580ed280fb", size = 74361, upload-time = "2024-12-22T23:09:42.37Z" }, + { url = "https://files.pythonhosted.org/packages/d2/a6/194b3625a7245c532ad418130d63077ce6cd241152524152f533e4d6edb0/tree_sitter_python-0.23.6-cp39-abi3-macosx_11_0_arm64.whl", hash = "sha256:680b710051b144fedf61c95197db0094f2245e82551bf7f0c501356333571f7a", size = 76436, upload-time = "2024-12-22T23:09:43.566Z" }, + { url = "https://files.pythonhosted.org/packages/d0/62/1da112689d6d282920e62c40e67ab39ea56463b0e7167bfc5e81818a770e/tree_sitter_python-0.23.6-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8a9dcef55507b6567207e8ee0a6b053d0688019b47ff7f26edc1764b7f4dc0a4", size = 112060, upload-time = "2024-12-22T23:09:44.721Z" }, + { url = "https://files.pythonhosted.org/packages/5d/62/c9358584c96e38318d69b6704653684fd8467601f7b74e88aa44f4e6903f/tree_sitter_python-0.23.6-cp39-abi3-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:29dacdc0cd2f64e55e61d96c6906533ebb2791972bec988450c46cce60092f5d", size = 112338, upload-time = "2024-12-22T23:09:48.323Z" }, + { url = "https://files.pythonhosted.org/packages/1a/58/c5e61add45e34fb8ecbf057c500bae9d96ed7c9ca36edb7985da8ae45526/tree_sitter_python-0.23.6-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:7e048733c36f564b379831689006801feb267d8194f9e793fbb395ef1723335d", size = 109382, upload-time = "2024-12-22T23:09:49.49Z" }, + { url = "https://files.pythonhosted.org/packages/e9/f3/9b30893cae9b3811fe652dc6f90aaadfda12ae0b2757f5722fc7266f423c/tree_sitter_python-0.23.6-cp39-abi3-win_amd64.whl", hash = "sha256:a24027248399fb41594b696f929f9956828ae7cc85596d9f775e6c239cd0c2be", size = 75904, upload-time = "2024-12-22T23:09:51.597Z" }, + { url = "https://files.pythonhosted.org/packages/87/cb/ce35a65f83a47b510d8a2f1eddf3bdbb0d57aabc87351c8788caf3309f76/tree_sitter_python-0.23.6-cp39-abi3-win_arm64.whl", hash = "sha256:71334371bd73d5fe080aed39fbff49ed8efb9506edebe16795b0c7567ed6a272", size = 73649, upload-time = "2024-12-22T23:09:53.71Z" }, +] + [[package]] name = "typing-extensions" version = "4.14.1"