foldforge-pylib

Shared Python libraries for the FoldForge GPU sidecars, published as one wheel (foldforge-storage, currently v0.3.0) so the four sidecars (rfdiffusion, proteinmpnn, boltz, af2) don't each carry a copy. The wheel ships three importable packages:

import	what
foldforge_storage	content-addressed artifact store (S3/R2, dedup by sha256)
foldforge_subprocess	cancellable GPU-subprocess runner (process-group kill, no zombie)
foldforge_trace	W3C traceparent extraction to bind a trace-id into sidecar logs

Sidecars pin a git tag: foldforge-storage @ git+ssh://git@github.com/FoldForge/foldforge-pylib.git@v0.3.0.

Install

pip install -e .            # core: storage compute-only + subprocess + trace
pip install -e ".[s3]"      # + boto3 for real R2/S3 upload/download
pip install -e ".[dev]"     # + pytest/ruff

foldforge_storage — content-addressed artifacts

from foldforge_storage import ArtifactStore

store = ArtifactStore(bucket="foldforge")   # reads FOLDFORGE_R2_ENDPOINT from env
art = store.put(pdb_bytes, "chemical/x-pdb")
# art.uri = r2://foldforge/artifacts/<sha256>.pdb, art.sha256, art.size_bytes

• FOLDFORGE_R2_ENDPOINT set → bytes are uploaded (deduped by content hash; HEAD-checked before PUT to skip existing).
• unset → compute-only: real sha256/size are still computed, nothing uploaded.
• Credentials come from the standard AWS env (AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY) via boto3 — never hardcoded.

foldforge_subprocess — cancellable GPU subprocess (DEBT #M2)

from foldforge_subprocess import run_cancellable, RunCancelled

run_cancellable(cmd, log_dir=work, should_cancel=lambda: not ctx.is_active())

Runs the GPU tool in its own process group; on should_cancel() it SIGTERMs the whole group (killing CUDA/torch grandchildren), waits a grace period, then SIGKILLs. stdout/stderr go to files (no PIPE deadlock), the child is always reaped (no zombie), and a clean completion is handled without a wasted kill. Raises RunCancelled on cancel, CalledProcessError on a non-zero exit.

foldforge_trace — trace-id binding (DEBT #M5)

Extracts the W3C traceparent from inbound gRPC metadata and binds the trace-id into the sidecar's structlog contextvars, so one request's logs across the gateway, orchestrator, and sidecars all share a single id.

Test

pip install -e ".[dev]"
pytest                                  # storage compute-only + subprocess + trace
FF_TEST_S3_ENDPOINT=http://127.0.0.1:9000 pytest   # + live S3 (MinIO) tests

Library code uses stdlib logging, not a logging framework — a shared lib must not impose one on its consumers.