Repository navigation
docs: document how to run the tests in tests/README.md - #28
Conversation
marcos-mendez
left a comment
There was a problem hiding this comment.
Hi @Nandika-Gupta, thank you, and welcome to Keel Linux!
I followed your instructions on a clean checkout and everything works exactly as written: 924 passed and 7 skipped, coverage at 100% with --fail-under=100 passing, and -k static6 selecting the IPv6 tests. Your facts are right too: CI runs Python 3.13 on ubuntu-latest, the 7 skips are exactly the ones you list for the reasons you give, and every test file is covered in the layout section. Going beyond the issue to explain what conftest.py stubs, and the Codespaces /tmp ACL pitfall, is the kind of detail that saves the next person an hour.
One change before merging, about keeping docs true over time:
Avoid copying facts that live elsewhere. The README repeats the module list from package: in .github/workflows/tests.yml, and the number of skipped tests. When someone adds a module or a skip, the code changes but the README doesn't, and nobody notices because nothing fails. Two common ways out:
- Point to the source of truth instead of copying it. For example: "use the module list from
package:in.github/workflows/tests.yml", and drop the exact count ("a few tests skip unless a real keel is present;pytest -rslists them"). - Or derive it in the command itself, so it can't drift.
Once that's in and CI is green, I'll merge. Thanks again!
|
@marcos-mendez Thanks for the review, that makes sense! |
|
Merged, thanks @Nandika-Gupta! Reading the module list straight from |
Closes #26
Adds
tests/README.md: what is needed, the commands CI runs, the 100 percentthreshold, the tests that skip without keel, notes on running outside CI
(Linux only; a
/tmpwith an ACL, as in GitHub Codespaces), and one line pergroup of test files. Adds a "Tests" section to
README.rstthat links to it.I followed the commands from a fresh checkout in GitHub Codespaces. There
25 tests failed with "writable by group or others":
/tmpcarries an ACLthat makes new files mode 646 whatever the umask, so the Keel Cloud flag the
tests write is refused. With
--basetemp=$HOME/pytmp/runthe run gives924 passed, 7 skipped, coverage 100 percent. The README says so.