✨ Add intermediate rewards with paper semantics - #799
Draft
flowerthrower wants to merge 10 commits into
Draft
flowerthrower wants to merge 10 commits into
flowerthrower wants to merge 10 commits into
Conversation
12 tasks
flowerthrower
force-pushed
the
v3/configurable-pass-timeouts
branch
from
August 27, 2026 14:46
9446b53 to
f8f444f
Compare
flowerthrower
force-pushed
the
v3/670-intermediate-rewards
branch
2 times, most recently
from
August 27, 2026 15:02
a25d10f to
21e573a
Compare
flowerthrower
force-pushed
the
v3/configurable-pass-timeouts
branch
2 times, most recently
from
August 27, 2026 15:07
0302862 to
d3609ee
Compare
flowerthrower
force-pushed
the
v3/670-intermediate-rewards
branch
from
August 27, 2026 15:07
21e573a to
1fbe20f
Compare
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
12 tasks
flowerthrower
force-pushed
the
v3/configurable-pass-timeouts
branch
from
August 28, 2026 07:33
d3609ee to
db2cee4
Compare
flowerthrower
added a commit
that referenced
this pull request
Aug 28, 2026
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
flowerthrower
force-pushed
the
v3/670-intermediate-rewards
branch
from
August 28, 2026 07:33
1fbe20f to
3905e40
Compare
flowerthrower
force-pushed
the
v3/configurable-pass-timeouts
branch
from
August 28, 2026 07:52
db2cee4 to
7606d81
Compare
flowerthrower
added a commit
that referenced
this pull request
Aug 28, 2026
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
flowerthrower
force-pushed
the
v3/670-intermediate-rewards
branch
2 times, most recently
from
August 28, 2026 08:13
462ea5c to
894d842
Compare
flowerthrower
force-pushed
the
v3/configurable-pass-timeouts
branch
from
August 28, 2026 08:30
5e9d1d7 to
837159b
Compare
flowerthrower
added a commit
that referenced
this pull request
Aug 28, 2026
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
flowerthrower
force-pushed
the
v3/670-intermediate-rewards
branch
from
August 28, 2026 08:30
894d842 to
04224ef
Compare
flowerthrower
force-pushed
the
v3/configurable-pass-timeouts
branch
from
August 28, 2026 09:57
837159b to
dd6fd15
Compare
flowerthrower
added a commit
that referenced
this pull request
Aug 28, 2026
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
flowerthrower
force-pushed
the
v3/670-intermediate-rewards
branch
3 times, most recently
from
August 28, 2026 10:18
27372b1 to
8c2a30f
Compare
flowerthrower
force-pushed
the
v3/configurable-pass-timeouts
branch
from
August 28, 2026 10:18
e802a3f to
3943b1b
Compare
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Assisted-by: GPT-5.6 via Codex Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
flowerthrower
force-pushed
the
v3/670-intermediate-rewards
branch
from
August 28, 2026 10:33
8c2a30f to
623b4ab
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 AI text below 🤖
Description
Adds configurable intermediate rewards while preserving the existing
PredictorEnvlifecycle and matching the paper prototype's reward behavior.The public controls are
intermediate_reward=True,reward_scale=1.0, andno_effect_penalty=-0.001. Settingintermediate_reward=Falseretains terminal-only rewards.Comparable expected-fidelity and estimated-success-probability steps use exact or approximate figure-of-merit deltas. Terminal ESP uses ASAP scheduling and stops charging a qubit after its final measurement or reset. Critical-depth actions use exact non-terminal deltas, while estimated Hellinger distance remains neutral until termination. Structural progress and structural no-effect steps remain neutral; optimization no-effect and failed actions receive
no_effect_penalty, and failures truncate the episode.This is position 8 of the stack. It depends on #798 and is followed by #800. No new package dependencies are introduced.
Fixes #670
Checklist
If PR contains AI-assisted content:
🤖 *AI text below* 🤖(titles are exempt).