fix(redteam): separate Mischievous User token usage

[jameshiester-oai]

Summary

• attribute Mischievous User simulator calls to attacker usage while keeping target calls and probes separate
• preserve billed usage from simulated-user and target error responses
• exclude Promptfoo response-cache hits without dropping provider-side cached-input usage
• keep the generic Simulated User provider's combined accounting behavior unchanged

Verification

• pnpm vitest run test/test-hygiene.test.ts test/providers/promptfoo.test.ts test/providers/simulatedUser.test.ts test/redteam/providers/mischievousUser.test.ts (211 tests)
• pnpm f && pnpm l
• pnpm tsc --noEmit
• pnpm build

Related to #10408.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a5f904d841

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.01%. Comparing base (4ba76df) to head (2c36a7b).

Additional details and impacted files

@@           Coverage Diff           @@
##             main   #10431   +/-   ##
=======================================
  Coverage   82.00%   82.01%           
=======================================
  Files         946      946           
  Lines       79198    79206    +8     
  Branches    26254    26256    +2     
=======================================
+ Hits        64950    64964   +14     
+ Misses      14248    14242    -6     

Flag	Coverage Δ
backend	83.44% <100.00%> (+<0.01%)	⬆️
site	23.24% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[promptfoo-scanner]

👍 All Clear

I reviewed the changes to the simulated user and redteam mischievous user providers, focusing on LLM data flows, prompt handling, and capability changes. The PR primarily refactors token usage accounting and error propagation without modifying prompts, tool access, or external I/O. Based on the diff, there are no new privilege boundaries crossed or execution sinks introduced. No LLM security issues were identified.

_{Minimum severity threshold: 🟡 Medium | To re-scan after changes, comment @promptfoo-scanner
Learn more}

---

_{Was this helpful?  👍 Yes  |  👎 No} 

[Copilot]

Pull request overview

This PR refines token-usage accounting for the redteam “Mischievous User” strategy by separating simulated-user (attacker) usage from target usage, while also avoiding charging Promptfoo response-cache hits.

Changes:

• Add a SimulatedUser hook to accumulate simulated-user vs. target token usage independently (and skip Promptfoo response-cache hits).
• Override the hook in RedteamMischievousUserProvider to attribute simulated-user calls to attacker usage.
• Add focused tests covering attacker/target separation, error paths, and cache-related behavior.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
test/redteam/providers/mischievousUser.test.ts	Adds coverage for attacker vs. target usage separation, error handling, and cache semantics.
src/redteam/providers/mischievousUser.ts	Overrides simulated-user token accumulation so Mischievous User usage is attributed to attacker usage.
src/providers/simulatedUser.ts	Introduces accumulation hooks and skips Promptfoo response-cache hits when accounting tokens/requests.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[promptfoo-scanner]

👍 All Clear

I reviewed changes to the simulated user/agent providers and redteam mischievous user provider. The PR primarily refactors token usage accounting and error propagation without altering prompts, capabilities, or execution sinks. Based on the diff, there are no new LLM security-relevant data flows or expanded privileges. No vulnerabilities were identified.

_{Minimum severity threshold: 🟡 Medium | To re-scan after changes, comment @promptfoo-scanner
Learn more}

---

_{Was this helpful?  👍 Yes  |  👎 No}