Add documentation using sphinx and github pages (Codex)

.github/workflows/docs_pages.yml

@@ -0,0 +1,74 @@
+name: Documentation Pages
+
+on:
+  push:
+    branches:
+      - main
+    tags:
+      - "v*"
+    paths:
+      - doc/**
+      - pyproject.toml
+      - .github/workflows/docs_pages.yml
+      - pyTorchAutoForge/**
+  pull_request:
+    branches:
+      - main
+    paths:
+      - doc/**
+      - pyproject.toml
+      - .github/workflows/docs_pages.yml
+      - pyTorchAutoForge/**
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+
+concurrency:
+  group: pages
+  cancel-in-progress: false
+
+jobs:
+  build-docs:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+          cache: pip
+      - name: Install docs dependencies
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install -e ".[docs]"
+      - name: Build Sphinx site
+        if: github.event_name == 'pull_request'
+        run: python -m sphinx -b html doc site
+      - name: Build versioned Sphinx site
+        if: github.event_name != 'pull_request'
+        run: bash doc/build_versioned_docs.sh site
+      - name: Configure GitHub Pages
+        if: github.event_name != 'pull_request'
+        uses: actions/configure-pages@v5
+      - name: Upload Pages artifact
+        if: github.event_name != 'pull_request'
+        uses: actions/upload-pages-artifact@v3
+        with:
+          path: site
+
+  deploy-docs:
+    if: github.event_name != 'pull_request'
+    needs: build-docs
+    runs-on: ubuntu-latest
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - name: Deploy to GitHub Pages
+        id: deployment
+        uses: actions/deploy-pages@v4

.gitignore

@@ -80,6 +80,8 @@ instance/
 
 # Sphinx documentation
 docs/_build/
+doc/_build/
+site/
 
 # PyBuilder
 .pybuilder/
@@ -173,3 +175,8 @@ cython_debug/
 .vscode/browse.vc.db
 .vscode/browse.vc.db-shm
 .vscode/browse.vc.db-wal
+CONTEXT.md
+
+# Documentation ignore
+doc/_autoapi_templates/*
+doc/_static/*

README.md

@@ -4,6 +4,24 @@
 
 A library based on PyTorch (<https://pytorch.org/>) and designed to automate ML models development, tracking and deployment, integrated with MLflow and Optuna (<https://mlflow.org/>, <https://optuna.org/>). It also supports spiking networks libraries (WIP). Model optimization and deployment can be performed using ONNx, pyTorch facilities or TensorRT (WIP). The library aims to be compatible with Jetson Orin Nano Jetpack rev6.1. Several other functionalities and utilities for sklearn and pySR (<https://github.com/MilesCranmer/PySR>) are included (see README and documentation).
 
+## Documentation
+
+Documentation is built with Sphinx, the PyData theme, and auto-generated public API pages. It is published through GitHub Pages:
+<https://petercalifano.github.io/pyTorchAutoForge/>.
+
+Local preview:
+
+```bash
+python -m pip install -e ".[docs]"
+doc/makedoc.sh -a
+```
+
+Local strict build:
+
+```bash
+doc/makedoc.sh
+```
+
 ## Some brief usage guides (WIP)
 
 ### TensorRT exporter quick usage
@@ -25,21 +43,35 @@ Notes:
 - `PYTHON` mode requires the `tensorrt` Python package.
 - Default behavior avoids architecture-specific flags and is suitable for Jetson deployment workflows.
 
-## Installation using pip
+## Installation Using Pip
 
-The suggested installation method is through pip as the others are mostly intended for development and may not be completely up-to-date with the newest release versions.
-In whatever conda or virtual environment you like (preferably with a sufficiently new torch release, to install from pypi:
+The package is available on PyPI. In any conda or virtual environment with a suitable PyTorch release:
 
 ```bash
-pip install pyTorchAutoForge
+python -m pip install pyTorchAutoForge
 ```
 
-Or from a local copy of the repository (requires `hatch` module for the build):
+From a local checkout:
 
 ```bash
 cd pyTorchAutoforge
-pip install .
+python -m pip install .
 ```
 
-An automatic installation script `conda_install.sh` is provided and should work in most cases. Note that it will automatically create a new environment named **autoforge** and makes several assumptions about your environment.
-Dependencies for the core modules should be installed automatically using pip. However, this is currently not fully tested. Please open related issues.
+An automatic installation script `conda_install.sh` is provided for development installs. By default it uses an existing `autoforge` conda environment and installs only the core package dependencies:
+
+```bash
+./conda_install.sh --create-env --editable
+```
+
+Optional extras are explicit:
+
+```bash
+./conda_install.sh --with-test --with-docs --build-docs
+```
+
+Jetson/ARM installs skip x86-only dependencies through package markers. Provide board-specific PyTorch wheels when needed:
+
+```bash
+./conda_install.sh --jetson --pytorch-url <wheel-or-url> --torchvision-url <wheel-or-url>
+```

TODO

@@ -51,7 +51,7 @@
   - [x] Improve backbone configuration
 
 - v0.5.0
-  - [ ] Migrate documentation from Read The Docs to something more controllable and less restrictive
+  - [x] Migrate documentation from Read The Docs to GitHub Pages with MkDocs
   - [ ] [MAJOR] Restructure and expand augmentations module to correctly handle images and 1D vectors jointly
   - [ ] Implement 1d vector error models (from selected distributions)
   - [ ] Change random apply and structure of augs module for images (split based on type)
@@ -74,7 +74,7 @@
   - [ ] Add pySR conveniency module
   - [ ] Explore Hydra for configuration management from yml files
   - [ ] Add prototype of SHAP for CNNs
-  - [ ] Add prototype of model explainer for vector-to-vector regression using Captum (2 methods)
+  - [x] Add prototype of model explainer for vector-to-vector regression using Captum (2 methods)
   - [ ] Upgrade api.torch to save model "packages" with all info to run the model
   - [ ] Improve docstrings in src code
   - [ ] Implement prototype of K-fold cross-validation for training runs
@@ -94,7 +94,7 @@
 
 - [ ] Add type annotations for mypy
 
-- [ ] Automate documentation building process
+- [x] Automate documentation building process
 
 - [x] Implement CI/CD to run tests
 

bash_scripts/check_torch_availability.sh

@@ -0,0 +1,86 @@
+#!/bin/bash
+set -euo pipefail
+
+# Script variables
+ENV_NAME="${CONDA_ENV:-autoforge}"
+CONDA_EXE="${CONDA_EXE:-conda}"
+USE_CONDA=1
+REQUIRE_CUDA=0
+
+# Usage guide
+usage() {
+    cat <<'EOF'
+Usage: bash_scripts/check_torch_availability.sh [options]
+
+Options:
+  -e, --env-name NAME   Conda environment name (default: CONDA_ENV or autoforge)
+      --conda-exe PATH  Conda executable (default: conda)
+      --no-conda        Do not activate conda environment
+      --require-cuda    Return non-zero when CUDA is unavailable
+  -h, --help            Show this help
+EOF
+}
+
+# Parser loop
+while [[ $# -gt 0 ]]; do
+    case "$1" in
+        -e|--env-name)
+            ENV_NAME="$2"
+            shift 2
+            ;;
+        --conda-exe)
+            CONDA_EXE="$2"
+            shift 2
+            ;;
+        --no-conda)
+            USE_CONDA=0
+            shift
+            ;;
+        --require-cuda)
+            REQUIRE_CUDA=1
+            shift
+            ;;
+        -h|--help)
+            usage
+            exit 0
+            ;;
+        *)
+            echo "Unknown option: $1" >&2
+            usage >&2
+            exit 2
+            ;;
+    esac
+done
+
+# Source conda environment
+if [[ "${USE_CONDA}" -eq 1 ]]; then
+    CONDA_BASE="$("${CONDA_EXE}" info --base)"
+    # shellcheck source=/dev/null
+    source "${CONDA_BASE}/etc/profile.d/conda.sh"
+    conda activate "${ENV_NAME}"
+fi
+
+# Set environment variable to make CUDA errors easier to debug
+export CUDA_LAUNCH_BLOCKING="${CUDA_LAUNCH_BLOCKING:-1}"
+export PTAF_REQUIRE_CUDA="${REQUIRE_CUDA}"
+
+# Check PyTorch and CUDA availability with a python command
+python -c '
+import os
+import sys
+
+import torch
+
+cuda_available = torch.cuda.is_available()
+print("Torch version:", torch.__version__)
+print("CUDA availability:", cuda_available)
+
+if cuda_available:
+    device_index = torch.cuda.current_device()
+    print("Torch device props:", torch.cuda.get_device_properties(device_index))
+    tensor = torch.tensor(2.0, device="cuda").fill_(3.14)
+    print("Tensor created on CUDA:", tensor)
+elif os.environ.get("PTAF_REQUIRE_CUDA") == "1":
+    print("CUDA is not available", file=sys.stderr)
+    sys.exit(1)
+'