Repository navigation
feat(check): judge π0.5 checkpoints by the season's task-set layout (1.5.0) - #16
Merged
Merged
Conversation
…1.5.0) AXIS seasons keep norm stats under the AXIS asset, but every openpi checkpoint was judged by the LIBERO layout, so a correct AXIS checkpoint drew non_canonical_norm_stats and submit refused to pay: no AXIS miner could submit. Pick the openpi layout from the season's benchmark -- the live row in the gate before the fee, the miner.yaml snapshot in check -- via protocol 0.12.0's openpi_layout_for.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A miner on the AXIS season (kyleab) could not submit:
openroboto check/ the gate insubmitjudged his correct checkpoint by the LIBERO layout, drewnon_canonical_norm_stats(stats underassets/axis-v0.1-task501-runtime-v1/, where the AXIS evaluator reads them), andsubmitrefuses to pay on warnings.openroboto-protocol==0.12.0(openpi_layout_for,AXIS_LAYOUT,Competition.benchmark)layout_of(..., benchmark)returns the openpi layout for the season's task set; thesubmitgate passes the live row'sbenchmark,checkthe snapshot'sinitsnapshotsbenchmark;Settings.competition_benchmarkrules:line saysπ0.5 (openpi, AXIS)454 tests green (new: AXIS check, and an end-to-end submit gate that pays for an AXIS tree — red if the live
benchmarkis not passed); lint green. Version 1.5.0 + CHANGELOG.