Repository navigation
[tracer] Runs of edenprobe: reports of their own, ban candidates and the Probe pages - #46
Merged
Merged
Conversation
A report whose first log holds a run of edenprobe (ScarletNX's log of it, alone or in the Share Log zip, Eden's or rEden's log, or report.jsonl) keeps the machine, the emulator, the mode and every case, apart from the games, and the reply says the counts and the suites that failed. The detail of the service, net and pad cases and the net notes, which can name the player, is taken out of every kept log first; a run sent twice by one person is kept once, and the next start makes the reports kept before into runs. Signed-off-by: AO55 <AO55@reden-emu.dev>
Fixed rules over the runs of edenprobe: a case that fails on one driver's range of versions for several people and passes on other drivers is a candidate for ScarletNX's ban table, with a draft line in its format; one that fails on most drivers is ScarletNX's own; one that a build broke on a driver is a regression. The tests sit on each threshold, and a mutant run in CI changes the rules one way at a time to check they would notice. Signed-off-by: AO55 <AO55@reden-emu.dev>
probe/ draws each case that fails somewhere by machine (the emulator, the driver and its version, the system), each case and each driver on a page of its own, and the ban candidates with their draft lines, ScarletNX's own failures and the regressions; a run's report page gives its counts and the cases that did not pass. tracer-mcp reads the same JSON in probe_matrix, probe_case and ban_candidates. Signed-off-by: AO55 <AO55@reden-emu.dev>
tests/bench_probe keeps a thousand runs of 1069 cases on fifteen machines in a new database and says, at each mark, the database's size, the Probe pages' and the time the rules and the pages take, so the cost where Tracer runs is a number to read again rather than a guess. make test builds it. Signed-off-by: AO55 <AO55@reden-emu.dev>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Depends on #45, the zip PR (this branch is based on it).
A run of edenprobe sent to Tracer is now a report of its own, kept by machine (the emulator, the driver and its version, the system), so the site tells which of edenprobe's cases pass on which driver and emulator.
[tracer] Runs of edenprobe as reports of their own: a report whose first log holds a run (ScarletNX's log of it, alone or in the Share Log zip, whose entry with the run is the one kept; Eden's or rEden's log; or report.jsonl, the console's reference) keeps the machine, the emulator, edenprobe's version and mode and every case, apart from the games' lists; the reply gives the counts and the suites that failed. The detail of the service, net and pad cases and the net notes, which can name the player, is taken out of every kept log first; a run sent twice by one person is kept once; the next start makes the reports kept before into runs.[tracer] Ban candidates, ScarletNX's own failures and regressions: fixed rules over ScarletNX's runs (Eden's shown beside them): a case failing on one driver's range of versions in 3 reports from 2 people, and passing in 3 reports of 2 other drivers with builds within 7 days, is a candidate for ScarletNX's ban table, with a draft line in its format and its key left to a person; one failing on 3 drivers (or every one seen) is ScarletNX's own; one a build broke on a driver is a regression.scripts/test-rule-mutants.py, in CI, changes the rules 19 ways and checks the tests catch each.[tracer] The runs of edenprobe on the site and in tracer-mcp:probe/(the cases by machine, a page per case and per driver, the candidates), a run's counts on its report page, the same as JSON inapi/probe/, and the toolsprobe_matrix,probe_caseandban_candidates.[tracer] Measure what the runs of edenprobe cost: make bench: a thousand runs of 1069 cases take about 41 KB of database each (40 MB per thousand), 19 MB of pages and 8 MB of JSON; every Probe page is drawn again in 3.3 processor seconds at a thousand runs (Ryzen 5 7500F). A run's log costs what its file weighs.After the reinstall, the database goes to version 8 and the start summarizes the kept reports again.