✨ Add intermediate rewards with paper semantics - #782
Closed
flowerthrower wants to merge 41 commits into
Closed
flowerthrower wants to merge 41 commits into
flowerthrower wants to merge 41 commits into
Conversation
12 tasks
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
flowerthrower
force-pushed
the
v3/670-intermediate-rewards
branch
from
August 27, 2026 11:08
245f8c3 to
28fe747
Compare
flowerthrower
force-pushed
the
v3/paper-reward-parity
branch
from
August 27, 2026 11:08
0ef3cca to
7c65897
Compare
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
flowerthrower
added a commit
that referenced
this pull request
Aug 27, 2026
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
flowerthrower
force-pushed
the
v3/paper-reward-parity
branch
from
August 27, 2026 13:09
7c65897 to
5d711c5
Compare
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Assisted-by: GPT-5.6 via Codex Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
flowerthrower
force-pushed
the
v3/paper-reward-parity
branch
from
August 27, 2026 13:18
5d711c5 to
6dd26c4
Compare
flowerthrower
added a commit
that referenced
this pull request
Aug 27, 2026
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Co-authored-by: Antonio Tudisco <anto.tu98@hotmail.it> Assisted-by: GPT-5.6 via Codex Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Assisted-by: GPT-5.6 via Codex Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Assisted-by: GPT-5.6 via Codex Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Assisted-by: GPT-5 via Codex Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Assisted-by: GPT-5 via Codex Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Assisted-by: GPT-5.6 via Codex Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
Co-authored-by: Antonio Tudisco <anto.tu98@hotmail.it> Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Assisted-by: GPT-5.6 via Codex Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com> Assisted-by: GPT 5.6 via Codex
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
flowerthrower
force-pushed
the
v3/paper-reward-parity
branch
from
August 27, 2026 13:21
6dd26c4 to
c785d7a
Compare
Member
Author
|
🤖 AI text below 🤖 Superseded by #792 after combining intermediate rewards and paper reward semantics at the top of the reordered stack. |
flowerthrower
added a commit
that referenced
this pull request
Aug 27, 2026
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
flowerthrower
added a commit
that referenced
this pull request
Aug 27, 2026
Signed-off-by: flowerthrower <flowerthrower@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 AI text below 🤖
Description
Combines the intermediate-reward feature and the paper reward semantics in one top-of-stack PR while retaining the existing
PredictorEnvlifecycle.The public reward controls are
intermediate_reward=True,reward_scale=1.0, andno_effect_penalty=-0.001. Settingintermediate_reward=Falseretains terminal-only rewards.no_effect_penalty, with failures truncating the episode.This is position 11 and the top of the stack. It depends directly on the GNN PR #788 and absorbs the former standalone intermediate-reward PR #760. No action, GNN, dependency, or experiment changes are included.
Validation
add-individual-bqskit-passesfor approximate expected fidelity, approximate ESP, calibration maps, and terminal ESP;Fixes #670
Checklist
If PR contains AI-assisted content:
🤖 *AI text below* 🤖.