Merge pull request #1328 from Open-Source-Legal/fix/dockerfile-pip-retries

Harden Dockerfile pip against mid-stream download failures

Author: JSv4

CHANGELOG.md

@@ -22,6 +22,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Fixed
 
+- **Backend CI build aborted on transient SSL failure mid-wheel-download** (`compose/local/django/Dockerfile`, `compose/production/django/Dockerfile`): Backend CI run `24646911374` on commit `233a9b67` (push to `main`) failed in the `Build the Stack` step with `ssl.SSLError: [SSL] record layer failure (_ssl.c:2590)` at 22 MB of 60.4 MB while `pip wheel` was downloading `opencv-python-headless`, aborting the entire build and cascading skips through `Run DB Migrations`, `Verify Docker Containers`, and `pytest`. Pip's default `--retries 5` only covers connection setup and does not resume broken mid-stream downloads — that behaviour is gated on `--resume-retries`, added in pip 24.1 (env var `PIP_RESUME_RETRIES`). Added `ENV PIP_RETRIES=10 PIP_TIMEOUT=60 PIP_RESUME_RETRIES=5` to both the `python-build-stage` and `python-run-stage` of both Dockerfiles so every `pip install` / `pip wheel` invocation (wheel build, `--upgrade pip`, spacy model downloads) picks up the hardened settings. Verified the base image (`pytorch/pytorch:2.7.1-cuda12.6-cudnn9-runtime`) ships pip 25.1.1 and recognises all three env vars (`pip config list` reports `:env:.resume-retries='5'`, `pip install --help` shows `--timeout (default 60.0 seconds)`).
 - **`fullDatacellList` no-args path was unbounded** (Issue #1256, follow-up to PR #1235, `config/graphql/extract_types.py:131-148`): `ExtractType.resolve_full_datacell_list` capped the `limit` and `offset-only` branches at `MAX_FULL_DATACELL_LIST_LIMIT` but returned the entire queryset when called with no arguments. Authenticated callers hitting `fullDatacellList` directly (no `limit`, no `offset`) could bypass the payload bound. Collapsed all three code paths so every call — no-args, offset-only, or `limit`+`offset` — returns `qs[start : start + min(limit_or_max, MAX_FULL_DATACELL_LIST_LIMIT)]`. The embed UI is unaffected (it already passes `limit: 500`); direct API callers now receive at most 500 cells and must paginate via `offset` to walk the rest. Added regression test `test_full_datacell_list_no_args_capped_at_server_max` in `opencontractserver/tests/test_extract_queries.py` which creates 501 cells and asserts the no-args response returns exactly `MAX_FULL_DATACELL_LIST_LIMIT` while `datacellCount` reports the true total.
 
 ### Changed

compose/local/django/Dockerfile

@@ -9,6 +9,15 @@ FROM python as python-build-stage
 ARG BUILD_ENVIRONMENT=local
 ARG GITHUB_ACTIONS
 
+# Harden pip against transient network failures during large wheel downloads.
+# PIP_RESUME_RETRIES (pip 24.1+) resumes broken downloads; the others bump
+# connection retry count and socket timeout. Without these, a mid-stream SSL
+# drop while fetching a large wheel (e.g. opencv-python-headless) aborts the
+# entire build.
+ENV PIP_RETRIES=10 \
+    PIP_TIMEOUT=60 \
+    PIP_RESUME_RETRIES=5
+
 # Debugging line for build args
 RUN echo "GITHUB_ACTIONS: $GITHUB_ACTIONS"
 
@@ -56,6 +65,11 @@ ENV PYTHONUNBUFFERED 1
 ENV PYTHONDONTWRITEBYTECODE 1
 ENV BUILD_ENV ${BUILD_ENVIRONMENT}
 
+# Harden pip against transient network failures (see python-build-stage for context).
+ENV PIP_RETRIES=10 \
+    PIP_TIMEOUT=60 \
+    PIP_RESUME_RETRIES=5
+
 # CUDA-specific environment variables for optimal performance
 ENV CUDA_MODULE_LOADING=LAZY
 ENV TORCH_CUDA_ARCH_LIST="6.0;6.1;7.0;7.5;8.0;8.6;8.9;9.0"

compose/production/django/Dockerfile

@@ -6,6 +6,15 @@ FROM python as python-build-stage
 
 ARG BUILD_ENVIRONMENT=production
 
+# Harden pip against transient network failures during large wheel downloads.
+# PIP_RESUME_RETRIES (pip 24.1+) resumes broken downloads; the others bump
+# connection retry count and socket timeout. Without these, a mid-stream SSL
+# drop while fetching a large wheel (e.g. opencv-python-headless) aborts the
+# entire build.
+ENV PIP_RETRIES=10 \
+    PIP_TIMEOUT=60 \
+    PIP_RESUME_RETRIES=5
+
 # Install apt packages
 RUN apt-get update && apt-get install --no-install-recommends -y \
   # dependencies for building Python packages
@@ -50,6 +59,11 @@ ENV PYTHONUNBUFFERED 1
 ENV PYTHONDONTWRITEBYTECODE 1
 ENV BUILD_ENV ${BUILD_ENVIRONMENT}
 
+# Harden pip against transient network failures (see python-build-stage for context).
+ENV PIP_RETRIES=10 \
+    PIP_TIMEOUT=60 \
+    PIP_RESUME_RETRIES=5
+
 # CUDA-specific environment variables for optimal performance
 ENV CUDA_MODULE_LOADING=LAZY
 ENV TORCH_CUDA_ARCH_LIST="6.0;6.1;7.0;7.5;8.0;8.6;8.9;9.0"