Skip to content

docs: document how to run the tests in tests/README.md - #28

Merged
marcos-mendez merged 2 commits into
Keel-Linux:masterfrom
Nandika-Gupta:docs/tests-readme
Oct 6, 2026
Merged

marcos-mendez merged 2 commits into
Keel-Linux:masterfrom
Nandika-Gupta:docs/tests-readme

Conversation

@Nandika-Gupta

Copy link
Copy Markdown

Closes #26

Adds tests/README.md: what is needed, the commands CI runs, the 100 percent
threshold, the tests that skip without keel, notes on running outside CI
(Linux only; a /tmp with an ACL, as in GitHub Codespaces), and one line per
group of test files. Adds a "Tests" section to README.rst that links to it.

I followed the commands from a fresh checkout in GitHub Codespaces. There
25 tests failed with "writable by group or others": /tmp carries an ACL
that makes new files mode 646 whatever the umask, so the Keel Cloud flag the
tests write is refused. With --basetemp=$HOME/pytmp/run the run gives
924 passed, 7 skipped, coverage 100 percent. The README says so.

@marcos-mendez marcos-mendez left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Nandika-Gupta, thank you, and welcome to Keel Linux!

I followed your instructions on a clean checkout and everything works exactly as written: 924 passed and 7 skipped, coverage at 100% with --fail-under=100 passing, and -k static6 selecting the IPv6 tests. Your facts are right too: CI runs Python 3.13 on ubuntu-latest, the 7 skips are exactly the ones you list for the reasons you give, and every test file is covered in the layout section. Going beyond the issue to explain what conftest.py stubs, and the Codespaces /tmp ACL pitfall, is the kind of detail that saves the next person an hour.

One change before merging, about keeping docs true over time:

Avoid copying facts that live elsewhere. The README repeats the module list from package: in .github/workflows/tests.yml, and the number of skipped tests. When someone adds a module or a skip, the code changes but the README doesn't, and nobody notices because nothing fails. Two common ways out:

  • Point to the source of truth instead of copying it. For example: "use the module list from package: in .github/workflows/tests.yml", and drop the exact count ("a few tests skip unless a real keel is present; pytest -rs lists them").
  • Or derive it in the command itself, so it can't drift.

Once that's in and CI is green, I'll merge. Thanks again!

@Nandika-Gupta

Copy link
Copy Markdown
Author

@marcos-mendez Thanks for the review, that makes sense!
Updated: The command now reads the module list from package: in tests.yml instead of copying it.
I also removed the counts that could go stale (skipped tests, Codespaces failures, the 3.11 note).
Re-ran it on a clean checkout: 924 passed, 7 skipped, coverage 100%.

@marcos-mendez
marcos-mendez merged commit 1a7344d into Keel-Linux:master Oct 6, 2026
2 checks passed
@marcos-mendez

Copy link
Copy Markdown
Collaborator

Merged, thanks @Nandika-Gupta! Reading the module list straight from tests.yml was exactly the right fix: the README can't drift from CI now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Document how to run the tests in tests/README.md

2 participants