feat(fitness): comprehensive system fitness checks for worker startup validation - #472

Merged

deanq merged 88 commits into

mainfrom

deanq/ae-1586-worker-fitness-checks

Apr 8, 2026

Commits on Mar 13, 2026

docs: add flash-based e2e test design spec
deanq
committed
27ae232

Commits on Mar 14, 2026

docs: add flash-based e2e tests implementation plan
deanq
committed
5df47c1
chore: scaffold e2e test directory and fixture project
deanq
committed
f5b9b76
feat: add QB fixture handlers for e2e tests
deanq
committed
83cf285
feat: add LB fixture handler for e2e tests
deanq
committed
ffc2556
chore: register e2e pytest markers (qb, lb, cold_start)
deanq
committed
ba063bb
feat: add e2e conftest with flash_server lifecycle fixture
deanq
committed
462e57e
feat: add e2e tests for sync and async QB handlers
deanq
committed
ce36fb9
feat: add e2e tests for stateful worker persistence
deanq
committed
8cfbc08
feat: add e2e tests for SDK Endpoint client round-trip
deanq
committed
a739e01
feat: add e2e tests for async SDK Endpoint client
deanq
committed
e449879
feat: add e2e cold start benchmark test
deanq
committed
e20d41f
feat: add e2e tests for LB remote dispatch
deanq
committed
aaaedf3
feat: replace CI-e2e.yml with flash-based QB e2e tests
deanq
committed
ecec224
feat: add nightly CI workflow for full e2e suite including LB
deanq
committed
62a804a
fix: correct e2e test request format, error handling, and CI config
deanq
committed
4ad36c8
docs: update branch CLAUDE.md with implementation context
deanq
committed
8b6c404
refactor: address code quality review findings
deanq
committed
e0647b9
fix(ci): install editable runpod with deps before flash
deanq
committed
e908de1
fix(e2e): initialize runpod.api_key from env var for SDK client tests
deanq
committed
0110fca
fix(ci): exclude tests/e2e from default pytest collection
deanq
committed
9d152fe
fix(e2e): warm up QB endpoints before running tests
deanq
committed
663b55d
fix(e2e): remove incompatible tests and reduce per-test timeout
deanq
committed
3fb35e8
fix(e2e): increase http client timeout and fix error assertion
deanq
committed
11c1835
fix(ci): update unit test matrix to Python 3.10-3.12
deanq
committed
0e89ae9
fix(e2e): remove stateful handler tests incompatible with remote dispatch
deanq
committed
fb278e0
fix(tests): fix mock targets and cold start threshold in unit tests
deanq
committed
db5d6ab
fix(ci): add pytest-rerunfailures for flaky remote dispatch timeouts
deanq
committed
6b89edb
fix(e2e): remove flaky raw httpx handler tests
deanq
committed
936e133
fix(e2e): consolidate SDK tests to single handler to reduce flakiness
deanq
committed
6b47858
fix(e2e): remove autouse from patch_runpod_globals to prevent cold endpoint
deanq
committed
b48e945
fix(ci): surface flash provisioning logs in e2e test output
deanq
committed
ce751fc
fix(e2e): surface flash server stderr to CI output
deanq
committed
f2662e6
feat(e2e): inject PR branch runpod-python into provisioned endpoints
deanq
committed
61ff231
refactor(e2e): redesign e2e tests to provision mock-worker endpoints
deanq
committed
cc911ec
fix(e2e): add structured logging to provisioner and test execution
deanq
committed
a8aa2e2
feat(e2e): add endpoint cleanup after test session
deanq
committed
1cb3489
chore(ci): remove nightly e2e workflow
deanq
committed
dd74099

Commits on Mar 21, 2026

fix(e2e): address PR review feedback
deanq
committed
eb1f9c3
perf(e2e): run jobs concurrently and consolidate endpoints
deanq
committed
98ce4af
fix(e2e): use flash undeploy CLI for reliable endpoint cleanup
deanq
committed
856882a
feat(serverless): add worker fitness check system
deanq
committed
222bf9e
feat(serverless): add worker fitness check system
deanq
committed
0485246
docs: document fitness check system
deanq
committed
e484ec9
feat(build): add GPU test binary build infrastructure
deanq
committed
fc7c6ca
feat(serverless): implement GPU fitness check system
deanq
committed
fc4945f
build(dist): package GPU test binary for distribution
deanq
committed
a5b6a1d
test(serverless): add GPU fitness check tests
deanq
committed
fc89a12
test(performance): disable GPU check in cold start benchmarks
deanq
committed
b6b0ec1
docs(serverless): document GPU fitness check system
deanq
committed
3ed7e2d
fix(fitness): defer GPU check registration to avoid circular imports
deanq
committed
48769b5
fix(logging): use warn() instead of warning() for RunPodLogger
deanq
committed
3044e37
fix(logging): fix RunPodLogger.warning() call in rp_scale
deanq
committed
7888fa7
fix(gpu-fitness): correct import path for rp_cuda
deanq
committed
f19f75d
fix(test): correct mock patch target for binary path resolution test
deanq
committed
543e76c
build(gpu-binary): replace ARM binary with x86-64 compiled version
deanq
committed
1842c18
feat(system-fitness): add system resource fitness checks
deanq
committed
bcf5f88
refactor(fitness): integrate system fitness checks auto-registration
deanq
committed
d670fc3
build(deps): add psutil for system resource checking
deanq
committed
521fe88
test(system-fitness): add comprehensive test suite for system fitness checks
deanq
committed
61430e3
test(fitness): update fixtures to handle system checks auto-registration
deanq
committed
01b32e8
docs: document built-in system fitness checks with configuration
deanq
committed
f895ae3
feat(cuda-init): add CUDA device initialization fitness check
deanq
committed
b1a93c3
docs: document CUDA device initialization fitness check
deanq
committed
28fb1d8
chore: reduce minimum disk space requirement to 1GB
deanq
committed
7c5dcd3
fix(cuda): suppress nvidia-smi stderr on CPU-only workers
deanq
committed
641551b
fix(cuda): parse actual CUDA version from nvidia-smi, not driver version
deanq
committed
61b3a11
refactor(disk-check): use percentage-based disk space validation
deanq
committed
0485260
docs: update disk space check documentation for percentage-based validation
deanq
committed
5b37cf3
fix(disk-check): remove redundant /tmp check in containers
deanq
committed
2012b84
fix(tests): update CUDA tests to match implementation
deanq
committed
ef8a2f7
fix(fitness): address PR feedback on fitness checks system
deanq
committed
17cccc8
fix(fitness): resolve CodeQL code quality issues
deanq
committed
2b033e0
refactor(gpu-fitness): remove redundant is_available() call in fallback
deanq
committed
077e1b6
fix(fitness): resolve unresolved PR feedback comments
deanq
committed
87b247c
fix(fitness): resolve CodeQL and Copilot feedback comments
deanq
committed
ac75dd2
fix(fitness): remove unused mock variable assignments in tests
deanq
committed
3fabd53
fix(ruff): resolve all remaining linting errors
deanq
committed
67eabff
chore: remove .claude/CLAUDE.md from fitness checks PR
deanq
committed
26c845a
refactor(tests): reorganize fitness check tests into dedicated subdirectory
deanq
committed
5eb2ddd
fix(tests): resolve linting issues in fitness test reorganization
deanq
committed
d9ad5d0
feat(fitness): add timing instrumentation to fitness checks
deanq
committed
d061234
fix(fitness): resolve exception handling and timing instrumentation issues
deanq
committed
40c4e24
fix(fitness): modernize types, fix async blocking, and improve error handling
deanq
committed
bdaf844
fix(fitness): address CodeQL and Copilot PR review feedback
deanq
committed
d019b37

Commits on Mar 24, 2026

fix(fitness): merge main and address CodeQL findings
deanq
committed
f371fa0

Commits on Mar 26, 2026

chore: remove accidentally committed superpowers docs
deanq
committed
b6a5e62
chore: remove CLAUDE.md development artifact
deanq
committed
7119f58