Skip to content

🐛 End RL episodes at the hard pass horizon - #801

Draft
flowerthrower wants to merge 2 commits into
munich-quantum-toolkit:mainfrom
flowerthrower:fix/rl-horizon-termination
Draft

flowerthrower wants to merge 2 commits into
munich-quantum-toolkit:mainfrom
flowerthrower:fix/rl-horizon-termination

Conversation

@flowerthrower

@flowerthrower flowerthrower commented Aug 28, 2026

Copy link
Copy Markdown
Member

🤖 AI text below 🤖

Description

Treat exhaustion of PredictorEnv.max_steps as a terminal outcome rather than
a time-limit truncation.

Stable-Baselines3 bootstraps Gymnasium truncations with the critic value of the
terminal observation. The configured pass limit is the hard compilation and
deployment horizon, so continuation beyond it is unreachable. This change
retains the zero reward but returns terminated=True and truncated=False when
a non-termination action consumes the final slot. Pass failures remain
truncations.

AI assistance was used to inspect the environment and Stable-Baselines3 rollout
semantics and to prepare this focused patch.

Checklist

  • The pull request only contains commits that are focused and relevant to this change.
  • I have added appropriate tests that cover the new/changed functionality.
  • I have updated the documentation to reflect these changes.
  • I have added entries to the changelog for any noteworthy additions, changes, fixes, or removals.
  • I have added migration instructions to the upgrade guide (if needed).
  • The changes follow the project's style guidelines and introduce no new warnings.
  • The changes are fully tested and pass the CI checks.
  • I have reviewed my own code changes.

If PR contains AI-assisted content:

  • Any agent that created, edited, or submitted GitHub content was explicitly authorized for that scope, as required by our AI Usage Guidelines.
  • Every agent-authored or agent-edited public text body begins with the visible disclosure 🤖 *AI text below* 🤖 (titles are exempt).
  • I have disclosed AI assistance in the PR description.
  • I confirm that I have personally reviewed and understood all AI-generated content, and accept full responsibility for it.

Assisted-by: GPT-5.6 via Codex

Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
Assisted-by: GPT 5.6 via Codex
Assisted-by: GPT-5.6 via Codex

Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
Assisted-by: GPT 5.6 via Codex
@codecov

codecov Bot commented Aug 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant