Add random tests generator - #15
Conversation
|
This looks good. Our current test-suite is too small and mostly covers positive cases, while experience shows that we should focus on negative ones. Auto-generation is a nice shortcut for that, and we can always move useful tests to the permanent testsuite. Is there a way to stop it from generating operations inside the main thread? It's really a place for declaration of domains and final assertions, and tracing is off there. Likewise for nested spawns (although this one we should fix at some point). |
|
Hm yeah, we could pass some arguments to |
An attempt at improving the test coverage:
Currently this is all a bit cheap! The program generator could produce more interesting logic. The random sampling is also super naïve since it favors "easy to find" outcomes, but that seems exhaustive enough at the moment (since dscheck rarely discovers more endings, the programs are too simple to hide anything).
I initially wrote this to test #3 but haven't found any bugs yet. I did confirm that it would have found the issues in the previous version of this PR :) What I did not expect is that it also produces broken programs on
main, I thought this branch was super exhaustive and rock solid. I've extracted the buggy cases in a new testsuite if someone wants to take a look, it's a bit weird.