Update README makefile commands for corporate macs

.development.env

@@ -1,4 +1,4 @@
 LOG_LEVEL="INFO"
 AJV_VALIDATOR_SCHEME=http
 AJV_VALIDATOR_HOST=localhost
-AJV_VALIDATOR_PORT=5002
+AJV_VALIDATOR_PORT=5005

.github/workflows/pull_request.yml

@@ -141,7 +141,7 @@ jobs:
 
       - name: Spin Up AJV Validator
         run: |
-          docker run --network validator-network --name ajv-validator -d -p 5002:5002 ${{ secrets.GAR_LOCATION }}/${{ secrets.GAR_PROJECT_ID }}/docker-images/eq-questionnaire-validator-ajv:$TAG
+          docker run --network validator-network --name ajv-validator -d -p 5005:5005 ${{ secrets.GAR_LOCATION }}/${{ secrets.GAR_PROJECT_ID }}/docker-images/eq-questionnaire-validator-ajv:$TAG
       - name: Spin Up Python Validator
         run: >
           docker run
@@ -162,7 +162,7 @@ jobs:
       - name: Check AJV Validator is Running
         run: |
           for i in {1..10}; do
-            if curl -f http://localhost:5002/status; then
+            if curl -f http://localhost:5005/status; then
               echo "AJV Validator is up!"
               exit 0
             fi

Dockerfile

@@ -17,7 +17,7 @@ COPY api.py poetry.lock pyproject.toml /usr/src/
 
 ENV AJV_VALIDATOR_SCHEME=http
 ENV AJV_VALIDATOR_HOST=localhost
-ENV AJV_VALIDATOR_PORT=5002
+ENV AJV_VALIDATOR_PORT=5005
 
 RUN poetry install --only main
 

Dockerfile-ajv

@@ -11,7 +11,7 @@ COPY schemas ../schemas
 
 RUN npm install --production=true
 
-ENV AJV_VALIDATOR_PORT=5002
+ENV AJV_VALIDATOR_PORT=5005
 ENV DEBUG=ajv-schema-validator
 
 WORKDIR /usr/src

README.md

@@ -4,42 +4,82 @@ An API for validating survey schemas.
 
 ## Setup
 
-In order to run locally you'll need Node.js, Poetry and Python installed.
-It's recommended that Python is installed via pyenv but pyenv is optional.
+In order to run locally you'll need Node.js, Poetry and Python installed, managed via a conda
+environment.
 
-### Install NVM and pyenv
+### Pre-Requisites
 
-NVM and pyenv will manage your versions of Node and Python and these commands will install the
-required versions of them which will be read from `.nvmrc` and `.python-version`.
+The following must be installed and working before you start:
+
+- Miniconda: Python, Node and system package management (install from Self Service)
+- Podman: Container runtime for supporting services (machine created and running)
+- gcloud: Pulling images from Google Artifact Registry
+
+Verify each is available:
+
+```shell
+conda --version
+podman --version
+gcloud --version
+```
+
+If `conda` reports `command not found` after installing from Self Service, the installer did not
+write the conda block into `~/.zshrc`. Confirm the install is present and wire it in:
 
 ```shell
-brew install nvm pyenv
-nvm use
-pyenv install
+ls -d /opt/miniconda3
+/opt/miniconda3/bin/conda init zsh
 ```
 
-If you get a message in the command line after running `nvm use` that the version of Node specified
-in the `.nvmrc` file isn't installed, just follow the commands to install it.
+Open a new terminal tab and re-check `conda --version`.
 
-e.g.
+### Conda environment
+
+Python and Node.js versions are pinned in the committed `environment.yml`, matching
+`.python-version` and `.nvmrc` as closely as conda-forge availability allows:
+
+> Note: conda-forge does not publish every Node patch release (it jumps from `22.13.0` to
+> `22.17.0`). Where the exact `.nvmrc` version is unavailable, pin the closest available patch
+> below it and note the substitution in `environment.yml`.
+
+If `.python-version` or `.nvmrc` change, update `environment.yml` to match.
+
+Create and activate the environment:
 
 ```shell
-nvm install v22.15.0
+conda env create -f environment.yml
+conda activate eq-validator
 ```
 
-### Install JS dependencies
+Version can be changed by editing `environment.yml` and running:
 
 ```shell
-npm install
+conda env update -f environment.yml --prune
 ```
 
-### Install Poetry and Python dependencies
+### Install dependencies
+
+With the conda environment active, install the Python dependencies:
 
 ```shell
-curl -sSL https://install.python-poetry.org | python3 - --version 2.1.2
 poetry install
 ```
 
+Install the JavaScript dependencies:
+
+```shell
+npm ci
+```
+
+Temporary fix, this section can be removed once conda-forge has a nodejs build bundling npm 11:
+conda-forge's `nodejs=22.13.0` bundles npm 10.9.2. `make run` will not start the Python app on npm 10:
+`npm run start` holds the terminal despite the trailing `&`, so make never reaches `poetry run python api.py`.
+Upgrade npm after creating the environment:
+
+```shell
+npm install -g npm@11.6.1
+```
+
 ## Running locally
 
 To run the app:
@@ -48,7 +88,7 @@ To run the app:
 make run
 ```
 
-Validator runs on two ports, `5001` is the main validator app and `5002` is Ajv validator.
+Validator runs on two ports, `5001` is the main validator app and `5005` is Ajv validator.
 
 ### Validator
 
@@ -65,7 +105,7 @@ if you want to run the app locally using multiple server workers you need to set
 
 ### Ajv validator
 
-Ajv validator defaults to running on `http://localhost:5002`.
+Ajv validator defaults to running on `http://localhost:5005`.
 
 You can override this by setting the `AJV_VALIDATOR_SCHEME`, `AJV_VALIDATOR_HOST`,
 and `AJV_VALIDATOR_PORT` environment variables.
@@ -76,7 +116,7 @@ The defaults for these are:
 
 - `AJV_VALIDATOR_SCHEME` = http
 - `AJV_VALIDATOR_HOST` = localhost
-- `AJV_VALIDATOR_PORT` = 5002
+- `AJV_VALIDATOR_PORT` = 5005
 
 Alternatively, you can override the entire URL by setting the `AJV_VALIDATOR_URL` environment variable directly.
 (**Note**: These values are also defined in the Dockerfiles, so if you choose to run
@@ -124,7 +164,7 @@ and the Validator app itself). However, if you want to start Ajv individually, r
 make start-ajv
 ```
 
-This defaults to running on port `5002`, set `AJV_VALIDATOR_PORT` in your .env file if you need to change this.
+This defaults to running on port `5005`, set `AJV_VALIDATOR_PORT` in your .env file if you need to change this.
 
 Running the Ajv server returns either an empty json response when the questionnaire is valid,
 or a response containing an "errors" key.
@@ -148,7 +188,7 @@ To run the app's Python tests:
 make test-python
 ```
 
-Make sure you don't already have Ajv running on localhost:5002 by running `lsof -i tcp:5002` if you
+Make sure you don't already have Ajv running on localhost:5005 by running `lsof -i tcp:5005` if you
 do make a note of the PID (process identifier) and then run `kill -9 <PID>`, replacing `<PID>` with
 the process id from the previous command.
 
@@ -217,7 +257,8 @@ best practices and maintaining consistency across the repository without the nee
 MegaLinter examines various file types and tools, including GitHub Actions, Shell scripts, Dockerfile, etc. It is
 configured using the `.mega-linter.yml` file.
 
-To run MegaLinter, ensure you have **Docker** installed on your system.
+To run MegaLinter, ensure you have **Podman** installed and running on your system
+(see [Running with Docker](#running-with-docker) below for setup and the `docker` command shim).
 
 > Note: The initial run may take some time to download the Docker image. However, subsequent executions will be
 > considerably faster due to Docker caching.
@@ -230,24 +271,33 @@ make megalint
 
 ## Running with Docker
 
-To install Docker run:
+Install Podman for your system as the container runtime.
+
+Make sure the Podman machine is started every time you want to use container images:
 
 ```shell
-brew install docker
+podman machine start
 ```
 
-On MacOS install container runtimes, e.g. Colima:
+This repo's Makefile and commands below only use `docker run` (no compose), so provide a
+`docker` command that points at Podman:
 
 ```shell
-brew install colima
+mkdir -p ~/.local/bin
+ln -s "$(which podman)" ~/.local/bin/docker
+hash -r
 ```
 
-Make sure Colima is started every time you want to use Docker images:
+`~/.local/bin` must be on your `PATH`. Verify:
 
 ```shell
-colima start
+docker --version
 ```
 
+> Note: `eq-questionnaire-runner` pulls and runs these images via `docker-compose-schema-validator.yml`.
+> That repo's Makefile uses `docker-compose` (not `docker compose`), so if you also work in that
+> repo you additionally need `podman-compose`, installed with `conda install -c conda-forge podman-compose`.
+
 When PRs are merged in this repo there is a GitHub workflow that builds 2 Docker images one for Validator
 and one for the Ajv validator and then pushes them to our GAR in GCP.
 These images can then be pulled down and run locally with Docker.
@@ -267,7 +317,7 @@ docker run -it -p 5001:5001 europe-west2-docker.pkg.dev/ons-eq-ci/docker-images/
 - Ajv validator:
 
 ```shell
-docker run -it -p 5002:5002 europe-west2-docker.pkg.dev/ons-eq-ci/docker-images/eq-questionnaire-validator-ajv
+docker run -it -p 5005:5005 europe-west2-docker.pkg.dev/ons-eq-ci/docker-images/eq-questionnaire-validator-ajv
 ```
 
 To stop these containers you may need to use the `docker kill` command:
@@ -292,6 +342,6 @@ docker kill <CONTAINER_ID>
 | `LOG_LEVEL`            | Sets the minimum log level, can be set to `DEBUG` to increase this level                      | `INFO`                                                                        |
 | `AJV_VALIDATOR_SCHEME` | Sets the scheme for the URL that Ajv validator will run on                                    | `http`                                                                        |
 | `AJV_VALIDATOR_HOST`   | Sets the host for the URL that Ajv validator will run on                                      | `localhost`                                                                   |
-| `AJV_VALIDATOR_PORT`   | Sets the port for the URL that Ajv validator will run on                                      | `5002`                                                                        |
+| `AJV_VALIDATOR_PORT`   | Sets the port for the URL that Ajv validator will run on                                      | `5005`                                                                        |
 | `AJV_VALIDATOR_URL`    | Sets complete URL that Ajv validator will run on                                              | `<AJV_VALIDATOR_SCHEME>://<AJV_VALIDATOR_HOST>:<AJV_VALIDATOR_PORT>/validate` |
 | `VALIDATOR_VERSION`    | Sets the version of the validator, this is used in the response from the `/validate` endpoint | `0.0.0`                                                                       |

ajv/app.js

@@ -6,12 +6,12 @@
 
 const debug = Debug("ajv-schema-validator");
 const app = express();
-const AJV_VALIDATOR_PORT = process.env.AJV_VALIDATOR_PORT || 5002;
+const AJV_VALIDATOR_PORT = process.env.AJV_VALIDATOR_PORT || 5005;
 
 app.use(
   express.json({
     limit: "2Mb",
   }),
 );
 
 // Only start the server if we're not in a test environment, otherwise export the app for testing purposes

api.py

@@ -30,6 +30,7 @@
 import requests
 import structlog
 import uvicorn
+from dotenv import load_dotenv
 from fastapi import Body, FastAPI
 from fastapi.responses import JSONResponse, Response
 from requests import RequestException
@@ -45,6 +46,8 @@
 
 ALLOWED_REPO_OWNERS = {"ONSdigital"}
 
+load_dotenv()
+
 AJV_VALIDATOR_SCHEME = os.getenv("AJV_VALIDATOR_SCHEME")
 
 AJV_VALIDATOR_HOST = os.getenv("AJV_VALIDATOR_HOST")

environment.yml

@@ -0,0 +1,10 @@
+name: eq-validator
+channels:
+  - conda-forge
+dependencies:
+  - python=3.14.6
+  - nodejs=22.13.0 # closest available patch to .nvmrc's v22.15.0
+  - poetry=2.4.2
+  - pip
+variables:
+  POETRY_VIRTUALENVS_CREATE: "false"

pyproject.toml

@@ -18,6 +18,7 @@ referencing = "^0.37.0"
 jsonpath-ng = "1.8.0"
 starlette = "^1.0.0"
 requests = "^2.33.1"
+python-dotenv = "^1.2.3"
 
 [tool.poetry.group.dev.dependencies]
 pytest = "^9.0.3"