diff --git a/DONE.md b/DONE.md index 54566ad..9152f09 100644 --- a/DONE.md +++ b/DONE.md @@ -1,5 +1,32 @@ # DONE +## 2026-04-27T00:00:00 — Update Markdown Files for local.cfg Removal +Updated all markdown files to reflect the env-var-based configuration approach. +`dspace/README.md` NOTE corrected from "Kubernetes Secrets mounted as files" to +the accurate ConfigMap/env-var description. Confirmed no other markdown files +had stale `local.cfg` or `backend/config/` references. All tables validated +with `check_tables.py`. + +- [x] Update `dspace/README.md` NOTE: replace "Kubernetes Secrets mounted as files" with the correct ConfigMap/env-var description +- [x] Verify no other markdown files have stale references +- [x] Run `python3 dotpy/check_tables.py` on edited files +- [x] Verify with the developer that the task is complete + +## 2026-04-27T00:00:00 — Replace local.cfg with Docker Compose Environment Variables +Deleted `backend/local.cfg` and removed its `COPY` instruction from +`backend.dockerfile`. The three settings it contained — password-auth plugin +sequence, `ip.bioIPsRange1`, and `ip.bioIPsRange2` — are now expressed as +`__P__`-encoded `environment:` variables in the `backend` service of +`docker-compose.yml`, mirroring the Kubernetes ConfigMap pattern used in +production (`backend-cm.jsonnet`). `README.md` updated to remove all +`local.cfg` references and document the env-var approach. + +- [x] Add `plugin__P__sequence__P__org__P__dspace__P__authenticate__P__AuthenticationMethod` and IP-range placeholder env vars to the `backend` service in `docker-compose.yml` +- [x] Remove `COPY ./backend/local.cfg $DSPACE_INSTALL/config/local.cfg` from `backend.dockerfile` +- [x] Delete `backend/local.cfg` +- [x] Verify the current state of the project achieves the task goal +- [x] Verify with the developer that the task is complete + ## 2026-04-22T00:00:00 — Backend Bin Scripts README Documented all 58 scripts and data files in `backend/bin/` (excluding `README.md` itself and the CSV inventory sheets under `sheets/`). Initial diff --git a/README.md b/README.md index 00d37c7..178d50c 100644 --- a/README.md +++ b/README.md @@ -38,9 +38,9 @@ The `GITHUB_BRANCH` build argument (default: `umich`) controls which branch of t > > **What is reusable:** the multi-stage Dockerfile patterns, `docker-compose.yml` service structure, Makefile workflow, smoke-test suite (`tests/`), and GitHub Actions CI pipeline (`.github/workflows/ci.yml`) are general-purpose and straightforward to adapt. > -> **What is U-M-specific:** the source forks (`mlibrary/DSpace`, `mlibrary/dspace-angular`), the `GITHUB_BRANCH=umich` default, backend scripts in `backend/bin/`, and `backend/local.cfg`. +> **What is U-M-specific:** the source forks (`mlibrary/DSpace`, `mlibrary/dspace-angular`), the `GITHUB_BRANCH=umich` default, and backend scripts in `backend/bin/`. > -> To adapt this for your own institution, point `GITHUB_BRANCH` (or a fork of your own) at your customized DSpace source and replace `backend/local.cfg` with your own overrides. Note that `backend/local.cfg` is only copied into images built by the root `backend.dockerfile` (local dev and `ci.yml`); the `dspace/backend.dockerfile` used to build published production/staging images does **not** copy it — configuration for those environments is supplied at runtime via environment variables or mounted Kubernetes Secrets. +> To adapt this for your own institution, point `GITHUB_BRANCH` (or a fork of your own) at your customized DSpace source and adjust the `environment:` block of the `backend` service in `docker-compose.yml` to suit your setup. All DSpace configuration is supplied via environment variables at runtime — for local dev through `docker-compose.yml`, and for production/staging through Kubernetes ConfigMaps or Secrets. ## Building and running locally @@ -105,7 +105,7 @@ DSPACE_REST_HOST=backend - The backend container exposes port **8000** (JDWP remote debugger — root `backend.dockerfile` for local dev only) and port **8009** (AJP connector). Neither is mapped in `docker-compose.yml` by default. Add a port mapping to `docker-compose.yml` if you need to attach a remote debugger locally. - The `backend` service uses `depends_on` with `condition: service_healthy` for `db` and `solr`, and the `frontend` service waits for `backend` to be healthy, ensuring correct startup ordering without manual delays. - Use `make` targets (see [Makefile](Makefile)) for common workflows: `make build`, `make up`, `make down`, `make clean`. -- **`backend/local.cfg`** disables OIDC authentication for local dev and supplies placeholder values for `ip.bioIPsRange1` / `ip.bioIPsRange2`. `OidcAuthenticationBean` is still instantiated by Spring even when removed from the authentication plugin sequence, and its initialisation path calls `String.split()` on those properties unconditionally — omitting them causes a `NullPointerException` that returns HTTP 500 on every `/server/api` endpoint and the actuator. This file is only copied into images built by the root `backend.dockerfile` (local dev and `ci.yml`). The `dspace/backend.dockerfile`, used to build the production/staging images published to `ghcr.io`, does **not** copy it at all. +- **Backend configuration** is supplied entirely via `environment:` variables in `docker-compose.yml` (mirroring the Kubernetes ConfigMap pattern used in production). Key local-dev overrides: `plugin__P__sequence__P__org__P__dspace__P__authenticate__P__AuthenticationMethod` disables OIDC and enables password auth; `ip__P__bioIPsRange1` / `ip__P__bioIPsRange2` are set to non-routable CIDR placeholders (`192.0.2.0/24`) so that `OidcAuthenticationBean` — which calls `String.split()` on those properties unconditionally at startup — does not throw a `NullPointerException` on every `/server/api` request. In production/staging, real IP ranges and all other settings come from the Kubernetes ConfigMap. ## Integration Testing diff --git a/TODO.md b/TODO.md index c70b44d..539c82f 100644 --- a/TODO.md +++ b/TODO.md @@ -1,7 +1,6 @@ # TODO - ## Scrub Deleted `.cpt` Files from Git History The five encrypted config files (`backend/config/*.cpt`) and the production log (`backend/logs/dspace.log.2023-11-01`) were deleted from the working tree in diff --git a/backend.dockerfile b/backend.dockerfile index de91683..1cee021 100644 --- a/backend.dockerfile +++ b/backend.dockerfile @@ -94,7 +94,6 @@ RUN mkdir /root/.emacs.d # Install additional backend scripts COPY ./backend/init.el /root/.emacs.d/init.el COPY ./backend/bin/ $DSPACE_INSTALL/bin/ -COPY ./backend/local.cfg $DSPACE_INSTALL/config/local.cfg # The logs directory is already created by `ant init_installation` above, # so the explicit mkdir is redundant. Kept here (commented out) as a reminder diff --git a/backend/local.cfg b/backend/local.cfg deleted file mode 100644 index af1fb4e..0000000 --- a/backend/local.cfg +++ /dev/null @@ -1,25 +0,0 @@ -# local.cfg — local development overrides (never commit secrets here) -# -# DSpace loads this file last, so any property here overrides dspace.cfg. -# This file is copied into the image by the root backend.dockerfile (used by -# docker-compose for local development and the ci.yml smoke-test workflow). -# dspace/backend.dockerfile (used to build production/staging images pushed to -# ghcr.io) does NOT copy backend/config/ at all — production/staging -# configuration is supplied at runtime via environment variables or mounted -# Kubernetes Secrets. - -# Disable OIDC authentication for local dev. -# authentication.cfg enables OidcAuthentication by default, but OidcAuthenticationBean -# reads ip.bioIPsRange1/2 on EVERY request and crashes with NullPointerException if -# those properties are missing, and weblogin.lib.umich.edu is unavailable locally anyway. -plugin.sequence.org.dspace.authenticate.AuthenticationMethod = org.dspace.authenticate.PasswordAuthentication - -# OidcAuthenticationBean is still instantiated by Spring even when removed from the -# authentication plugin sequence above. Its initialisation / health-check path calls -# String.split() on the values of ip.bioIPsRange1 and ip.bioIPsRange2 unconditionally, -# so a NullPointerException propagates to every /server/api endpoint and the actuator -# when the properties are absent. Provide harmless non-routable placeholders so the -# bean has a non-null string to split while never matching any real request IP. -ip.bioIPsRange1 = 192.0.2.0/24 -ip.bioIPsRange2 = 192.0.2.0/24 - diff --git a/docker-compose.yml b/docker-compose.yml index 37ff2ce..22b7569 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -63,6 +63,15 @@ services: - dspace__P__server__P__url=http://backend:8080/server # dspace.ui.url is used by the backend to generate frontend links (e.g. e-mail alerts). - dspace__P__ui__P__url=http://localhost:4000 + # Disable OIDC authentication for local dev; use password auth only. + # In production/staging this is set via the external Kubernetes ConfigMap. + - plugin__P__sequence__P__org__P__dspace__P__authenticate__P__AuthenticationMethod=org.dspace.authenticate.PasswordAuthentication + # OidcAuthenticationBean is instantiated by Spring even when removed from the plugin + # sequence above. It calls String.split() on ip.bioIPsRange1/2 unconditionally, so + # provide harmless non-routable CIDR placeholders to prevent NullPointerException. + # In production/staging real IP ranges are supplied via the Kubernetes ConfigMap. + - ip__P__bioIPsRange1=192.0.2.0/24 + - ip__P__bioIPsRange2=192.0.2.0/24 ports: - "8080:8080" volumes: diff --git a/dspace/README.md b/dspace/README.md index 68067df..be2efa4 100644 --- a/dspace/README.md +++ b/dspace/README.md @@ -10,7 +10,7 @@ Run the GitHub action workflows in the following order to build the images that | [2. Build dspace images](https://github.com/mlibrary/dspace-containerization/actions/workflows/build-dspace-images.yml) | dspace-frontend:`tag`, dspace-backend:`tag`, dspace-solr:`tag`, dspace-db:`tag` | NOTE: -* Production configuration is **not** baked into the backend image. It is delivered at runtime via Kubernetes Secrets mounted directly as single files into the container (e.g. `dspace.cfg`, `authentication-oidc.cfg`). The actual Secret names and mount paths are managed in [deepblue-documents-kube](https://github.com/mlibrary/deepblue-documents-kube). +* Production configuration is **not** baked into the backend image. It is supplied at runtime entirely via environment variables — non-sensitive settings (DSpace property overrides, IP ranges, mail settings, etc.) from a Kubernetes **ConfigMap**, and sensitive values (credentials, API keys) from Kubernetes **Secrets**. Both are managed in [deepblue-documents-kube](https://github.com/mlibrary/deepblue-documents-kube). This mirrors the pattern used locally: the `backend` service in `docker-compose.yml` sets the same `__P__`-encoded env vars for local development. ## configuration and deployment Argo CD is used for configuration and deployment via [deepblue-documents-kube](https://github.com/mlibrary/deepblue-documents-kube) repository.