Skip to content

✨ Add intermediate rewards to RL - #760

Closed
flowerthrower wants to merge 20 commits into
v3/667-stochastic-actionsfrom
v3/670-intermediate-rewards
Closed

✨ Add intermediate rewards to RL#760
flowerthrower wants to merge 20 commits into
v3/667-stochastic-actionsfrom
v3/670-intermediate-rewards

Conversation

@flowerthrower

@flowerthrower flowerthrower commented Aug 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

🤖 AI text below 🤖

Description

Enables intermediate rewards in PredictorEnv by default. For expected_fidelity and estimated_success_probability, comparable non-terminal steps use changes in the exact or approximate figure of merit. Transitions between incomparable reward regimes remain neutral, and optimization actions with no measurable change receive a configurable penalty.

Set intermediate_reward=False to retain terminal-only rewards. reward_scale and no_effect_penalty configure the shaping behavior on PredictorEnv.

This stacked PR depends on #781. Validated with uvx nox -s lint.

Fixes #670

Checklist

  • The pull request only contains commits that are focused and relevant to this change.
  • I have added appropriate tests that cover the new/changed functionality.
  • I have updated the documentation to reflect these changes.
  • I have added entries to the changelog for any noteworthy additions, changes, fixes, or removals.
  • I have added migration instructions to the upgrade guide (if needed).
  • The changes follow the project's style guidelines and introduce no new warnings.
  • The changes are fully tested and pass the CI checks.
  • I have reviewed my own code changes.

If PR contains AI-assisted content:

  • Any agent that created, edited, or submitted GitHub content was explicitly authorized for that scope, as required by our AI Usage Guidelines.
  • Every agent-authored or agent-edited public text body begins with the visible disclosure 🤖 *AI text below* 🤖 (titles are exempt).
  • I have disclosed AI assistance in the PR description.
  • I confirm that I have personally reviewed and understood all AI-generated content, and accept full responsibility for it.

@flowerthrower
flowerthrower force-pushed the v3/670-intermediate-rewards branch from c4fe0c9 to 00caf9e Compare August 20, 2026 12:27
@flowerthrower
flowerthrower force-pushed the v3/670-intermediate-rewards branch from 00caf9e to 75d6d51 Compare August 21, 2026 15:34
@flowerthrower flowerthrower mentioned this pull request Aug 21, 2026
Closed
12 tasks
@flowerthrower
flowerthrower force-pushed the v3/670-intermediate-rewards branch from 75d6d51 to 4585e35 Compare August 21, 2026 15:55
@codecov

codecov Bot commented Aug 26, 2026 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 21.95122% with 32 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/mqt/predictor/rl/predictorenv.py 21.9% 32 Missing ⚠️

📢 Thoughts on this report? Let us know!

@flowerthrower
flowerthrower force-pushed the v3/670-intermediate-rewards branch from 6d8e162 to 9fd0051 Compare August 26, 2026 11:07
@flowerthrower
flowerthrower force-pushed the v3/670-intermediate-rewards branch from 9fd0051 to a307857 Compare August 26, 2026 11:28
@flowerthrower flowerthrower added enhancement New feature or request major Part of a major release labels Aug 26, 2026
@flowerthrower flowerthrower self-assigned this Aug 26, 2026
@flowerthrower
flowerthrower force-pushed the v3/675-qiskit-passes branch 2 times, most recently from be47293 to 1545b98 Compare August 26, 2026 13:43
This was referenced Aug 27, 2026
Merged
Merged
flowerthrower and others added 5 commits August 27, 2026 12:12
@flowerthrower
🎨 Add wrapper for stochastic passes
4a455e2 
Assisted-by: GPT-5 via Codex

Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
Assisted-by: GPT 5.6 via Codex
@flowerthrower
🎨 Score stochastic candidates in the target basis
c2bfd65 
Assisted-by: GPT-5 via Codex

Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
Assisted-by: GPT 5.6 via Codex
@pre-commit-ci @flowerthrower
🎨 pre-commit fixes
6073b40
@flowerthrower
📝 Document stochastic RL action changes
4e55bdc 
Assisted-by: GPT-5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
@flowerthrower
♻️ Preserve single-attempt layout handling
aa8e530 
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
Assisted-by: GPT 5.6 via Codex
@flowerthrower
flowerthrower force-pushed the v3/670-intermediate-rewards branch from a307857 to 245f8c3 Compare August 27, 2026 10:38
@flowerthrower flowerthrower mentioned this pull request Aug 27, 2026
Closed
6 tasks
@flowerthrower
flowerthrower changed the base branch from v3/675-qiskit-passes to v3/reproducible-rl-seeding August 27, 2026 10:50
@flowerthrower
🐛 Continue after failed stochastic attempts
303bb1e 
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
Assisted-by: GPT 5.6 via Codex
@flowerthrower
✨ Add SABRE routing action
67407d5 
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
Assisted-by: GPT 5.6 via Codex
@flowerthrower
📝 Document the SABRE routing action
7dca492 
Assisted-by: GPT-5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
flowerthrower and others added 12 commits August 27, 2026 13:07
@flowerthrower
✨ Expand Qiskit RL actions
1106dd1 
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
Assisted-by: GPT 5.6 via Codex
@flowerthrower
🧪 Align Qiskit action tests
4ad28bd 
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
Assisted-by: GPT 5.6 via Codex
@flowerthrower
🧪 Cover Qiskit actions through pass invariants
31346d2 
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
Assisted-by: GPT 5.6 via Codex
@flowerthrower
♻️ Assume bidirectional device coupling
9983def 
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
Assisted-by: GPT 5.6 via Codex
@flowerthrower
🎨 Treat SABRE routing as stochastic
f5868e0 
Assisted-by: GPT-5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
@flowerthrower
✨ Add QSD and MGD synthesis actions
dd87d7c 
Co-authored-by: Antonio Tudisco <anto.tu98@hotmail.it>

Assisted-by: GPT-5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
@flowerthrower
✨ Add TKET decomposition and placement actions
c57f281 
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
Assisted-by: GPT 5.6 via Codex
@flowerthrower
🐛 Honor Gymnasium seeds for circuit sampling
d730db4 
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
Assisted-by: GPT 5.6 via Codex
@flowerthrower
✨ Add intermediate rewards to RL
97934f4 
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
Assisted-by: GPT 5.6 via Codex
@pre-commit-ci @flowerthrower
🎨 pre-commit fixes
f06928c
@flowerthrower
🐛 Fix intermediate reward linting
443b627 
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
Assisted-by: GPT 5.6 via Codex
@flowerthrower
📝 Document intermediate RL rewards
28fe747 
Assisted-by: GPT-5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
@flowerthrower
flowerthrower force-pushed the v3/670-intermediate-rewards branch from 245f8c3 to 28fe747 Compare August 27, 2026 11:08
@flowerthrower
flowerthrower force-pushed the v3/reproducible-rl-seeding branch 2 times, most recently from d730db4 to 788afde Compare August 27, 2026 13:09
Base automatically changed from v3/reproducible-rl-seeding to v3/667-stochastic-actions August 27, 2026 13:10
@flowerthrower
flowerthrower force-pushed the v3/667-stochastic-actions branch 2 times, most recently from 3f75c42 to b78feea Compare August 27, 2026 13:21
@flowerthrower flowerthrower mentioned this pull request Aug 27, 2026
Closed
6 tasks
@flowerthrower

flowerthrower commented Aug 27, 2026

Copy link
Copy Markdown
Member Author

🤖 AI text below 🤖

Folded into the combined reward PR #792.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@flowerthrower flowerthrower

Labels

enhancement New feature or request major Part of a major release

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

✨ add step-wise intermediate rewards

1 participant

@flowerthrower