fix(metrics): evaluate_batch still calls the pre-refactor evaluate signatures

[Anai-Guo]

Description

Two evaluate_batch helpers in langtest/metrics/ were never updated when the methods they call were switched to dict-shaped inputs. Both raise TypeError on the first iteration.

1. RatingEval.evaluate_batch — langtest/metrics/llm_eval.py

def evaluate(self, inputs: dict, predictions: dict) -> List[dict]:     # line 432
...
def evaluate_batch(self, examples: List[dict]) -> List[dict]:          # line 469
    for example in examples:
        prompt = example.get("prompt", "")
        response = example.get("response", "")
        groundtruth = example.get("groundtruth") if self.include_groundtruth else None

        result = self.evaluate(prompt, response, groundtruth)          # 3 positional args

TypeError: RatingEval.evaluate() takes 3 positional arguments but 4 were given

SummaryEval.evaluate_batch, ~100 lines above in the same file, is the correct sibling — it already passes the two dicts:

result = self.evaluate(input_example, prediction_example)              # line 364

2. PrometheusEval.evaluate_batch — langtest/metrics/prometheus_eval.py

def evaluate_response(self, llm_response: Dict[str, str]) -> Tuple[str, int]:   # line 91
...
def evaluate_batch(self, entries: List[Dict[str, str]]) -> List[Tuple[str, int]]:
    queries = [entry.get("query", None) for entry in entries]
    results = [entry.get("result", None) for entry in entries]
    answers = [entry.get("answer", None) for entry in entries]
    return [
        self.evaluate_response(query, result, answer)                  # 3 positional args
        for query, result, answer in zip(queries, results, answers)
    ]

TypeError: PrometheusEval.evaluate_response() takes 2 positional arguments but 4 were given

This one takes out the public entry point too — PrometheusEval.evaluate() builds the entry dicts and delegates straight to evaluate_batch, so evaluate() cannot return.

The destructuring is also lossy: evaluate_response reads response_a / response_b off the entry for eval_type == "relative_grading", and those keys were being dropped.

Changes

• RatingEval.evaluate_batch: build the inputs / predictions dicts and pass them, matching SummaryEval.evaluate_batch. include_groundtruth gating is left to evaluate, which already applies it.
• PrometheusEval.evaluate_batch: forward each entry unchanged — evaluate_response already reads every key it needs, including response_a / response_b.

Verification

Signatures rebuilt from the repo files with ast, calls replayed through inspect.Signature.bind:

RatingEval.evaluate(self, inputs: dict, predictions: dict)
  main  self.evaluate(prompt, response, groundtruth)   -> TypeError: too many positional arguments
  fixed self.evaluate(inputs, predictions)             -> OK
PrometheusEval.evaluate_response(self, llm_response: Dict[str, str])
  main  self.evaluate_response(query, result, answer)  -> TypeError: too many positional arguments
  fixed self.evaluate_response(entry)                  -> OK

black==23.3.0 --line-length=90 reports both files unchanged, and flake8==6.0.0 produces byte-identical findings before and after the patch (all pre-existing, none on the touched lines).

🤖 Generated with Claude Code