From 83c41158d19d091bfd401bdc466fb296e3623dc1 Mon Sep 17 00:00:00 2001 From: Greg Kostin Date: Mon, 27 Apr 2026 20:25:34 -0400 Subject: [PATCH 1/3] Replace local.cfg with docker-compose environment variables - Delete backend/local.cfg - Remove COPY of local.cfg from backend.dockerfile - Add plugin sequence (PasswordAuthentication) and ip.bioIPsRange1/2 placeholder env vars to the backend service in docker-compose.yml, mirroring the __P__-encoded env-var pattern used by the Kubernetes ConfigMap in deepblue-documents-kube - Update README.md: remove local.cfg references; document env-var approach - Update dspace/README.md: replace stale "Secrets mounted as files" note with correct ConfigMap/env-var description --- README.md | 6 +++--- TODO.md | 21 +++++++++++++++++++++ backend.dockerfile | 1 - backend/local.cfg | 25 ------------------------- docker-compose.yml | 9 +++++++++ dspace/README.md | 2 +- 6 files changed, 34 insertions(+), 30 deletions(-) delete mode 100644 backend/local.cfg diff --git a/README.md b/README.md index 00d37c7..178d50c 100644 --- a/README.md +++ b/README.md @@ -38,9 +38,9 @@ The `GITHUB_BRANCH` build argument (default: `umich`) controls which branch of t > > **What is reusable:** the multi-stage Dockerfile patterns, `docker-compose.yml` service structure, Makefile workflow, smoke-test suite (`tests/`), and GitHub Actions CI pipeline (`.github/workflows/ci.yml`) are general-purpose and straightforward to adapt. > -> **What is U-M-specific:** the source forks (`mlibrary/DSpace`, `mlibrary/dspace-angular`), the `GITHUB_BRANCH=umich` default, backend scripts in `backend/bin/`, and `backend/local.cfg`. +> **What is U-M-specific:** the source forks (`mlibrary/DSpace`, `mlibrary/dspace-angular`), the `GITHUB_BRANCH=umich` default, and backend scripts in `backend/bin/`. > -> To adapt this for your own institution, point `GITHUB_BRANCH` (or a fork of your own) at your customized DSpace source and replace `backend/local.cfg` with your own overrides. Note that `backend/local.cfg` is only copied into images built by the root `backend.dockerfile` (local dev and `ci.yml`); the `dspace/backend.dockerfile` used to build published production/staging images does **not** copy it — configuration for those environments is supplied at runtime via environment variables or mounted Kubernetes Secrets. +> To adapt this for your own institution, point `GITHUB_BRANCH` (or a fork of your own) at your customized DSpace source and adjust the `environment:` block of the `backend` service in `docker-compose.yml` to suit your setup. All DSpace configuration is supplied via environment variables at runtime — for local dev through `docker-compose.yml`, and for production/staging through Kubernetes ConfigMaps or Secrets. ## Building and running locally @@ -105,7 +105,7 @@ DSPACE_REST_HOST=backend - The backend container exposes port **8000** (JDWP remote debugger — root `backend.dockerfile` for local dev only) and port **8009** (AJP connector). Neither is mapped in `docker-compose.yml` by default. Add a port mapping to `docker-compose.yml` if you need to attach a remote debugger locally. - The `backend` service uses `depends_on` with `condition: service_healthy` for `db` and `solr`, and the `frontend` service waits for `backend` to be healthy, ensuring correct startup ordering without manual delays. - Use `make` targets (see [Makefile](Makefile)) for common workflows: `make build`, `make up`, `make down`, `make clean`. -- **`backend/local.cfg`** disables OIDC authentication for local dev and supplies placeholder values for `ip.bioIPsRange1` / `ip.bioIPsRange2`. `OidcAuthenticationBean` is still instantiated by Spring even when removed from the authentication plugin sequence, and its initialisation path calls `String.split()` on those properties unconditionally — omitting them causes a `NullPointerException` that returns HTTP 500 on every `/server/api` endpoint and the actuator. This file is only copied into images built by the root `backend.dockerfile` (local dev and `ci.yml`). The `dspace/backend.dockerfile`, used to build the production/staging images published to `ghcr.io`, does **not** copy it at all. +- **Backend configuration** is supplied entirely via `environment:` variables in `docker-compose.yml` (mirroring the Kubernetes ConfigMap pattern used in production). Key local-dev overrides: `plugin__P__sequence__P__org__P__dspace__P__authenticate__P__AuthenticationMethod` disables OIDC and enables password auth; `ip__P__bioIPsRange1` / `ip__P__bioIPsRange2` are set to non-routable CIDR placeholders (`192.0.2.0/24`) so that `OidcAuthenticationBean` — which calls `String.split()` on those properties unconditionally at startup — does not throw a `NullPointerException` on every `/server/api` request. In production/staging, real IP ranges and all other settings come from the Kubernetes ConfigMap. ## Integration Testing diff --git a/TODO.md b/TODO.md index c70b44d..b49c5c5 100644 --- a/TODO.md +++ b/TODO.md @@ -1,6 +1,27 @@ # TODO +## Replace local.cfg with Docker Compose Environment Variables +Remove `backend/local.cfg` and its `COPY` instruction from `backend.dockerfile`; +express the same settings as `environment:` entries in the `backend` service of +`docker-compose.yml`, mirroring the pattern used by the Kubernetes ConfigMap at +`deepblue-documents-kube/environments/deepblue-documents/demo/backend-cm.jsonnet`. + +- [x] Add `plugin__P__sequence__P__org__P__dspace__P__authenticate__P__AuthenticationMethod` and IP-range placeholder env vars to the `backend` service in `docker-compose.yml` +- [x] Remove `COPY ./backend/local.cfg $DSPACE_INSTALL/config/local.cfg` from `backend.dockerfile` +- [x] Delete `backend/local.cfg` +- [x] Verify the current state of the project achieves the task goal +- [ ] Verify with the developer that the task is complete + + +## Update Markdown Files for local.cfg Removal +Update all markdown files to reflect the current project state after removing +`backend/local.cfg` and switching to env-var-based configuration. + +- [x] Update `dspace/README.md` NOTE: replace "Kubernetes Secrets mounted as files" with the correct ConfigMap/env-var description +- [x] Verify no other markdown files have stale references +- [x] Run `python3 dotpy/check_tables.py` on edited files +- [ ] Verify with the developer that the task is complete ## Scrub Deleted `.cpt` Files from Git History The five encrypted config files (`backend/config/*.cpt`) and the production log diff --git a/backend.dockerfile b/backend.dockerfile index de91683..1cee021 100644 --- a/backend.dockerfile +++ b/backend.dockerfile @@ -94,7 +94,6 @@ RUN mkdir /root/.emacs.d # Install additional backend scripts COPY ./backend/init.el /root/.emacs.d/init.el COPY ./backend/bin/ $DSPACE_INSTALL/bin/ -COPY ./backend/local.cfg $DSPACE_INSTALL/config/local.cfg # The logs directory is already created by `ant init_installation` above, # so the explicit mkdir is redundant. Kept here (commented out) as a reminder diff --git a/backend/local.cfg b/backend/local.cfg deleted file mode 100644 index af1fb4e..0000000 --- a/backend/local.cfg +++ /dev/null @@ -1,25 +0,0 @@ -# local.cfg — local development overrides (never commit secrets here) -# -# DSpace loads this file last, so any property here overrides dspace.cfg. -# This file is copied into the image by the root backend.dockerfile (used by -# docker-compose for local development and the ci.yml smoke-test workflow). -# dspace/backend.dockerfile (used to build production/staging images pushed to -# ghcr.io) does NOT copy backend/config/ at all — production/staging -# configuration is supplied at runtime via environment variables or mounted -# Kubernetes Secrets. - -# Disable OIDC authentication for local dev. -# authentication.cfg enables OidcAuthentication by default, but OidcAuthenticationBean -# reads ip.bioIPsRange1/2 on EVERY request and crashes with NullPointerException if -# those properties are missing, and weblogin.lib.umich.edu is unavailable locally anyway. -plugin.sequence.org.dspace.authenticate.AuthenticationMethod = org.dspace.authenticate.PasswordAuthentication - -# OidcAuthenticationBean is still instantiated by Spring even when removed from the -# authentication plugin sequence above. Its initialisation / health-check path calls -# String.split() on the values of ip.bioIPsRange1 and ip.bioIPsRange2 unconditionally, -# so a NullPointerException propagates to every /server/api endpoint and the actuator -# when the properties are absent. Provide harmless non-routable placeholders so the -# bean has a non-null string to split while never matching any real request IP. -ip.bioIPsRange1 = 192.0.2.0/24 -ip.bioIPsRange2 = 192.0.2.0/24 - diff --git a/docker-compose.yml b/docker-compose.yml index 37ff2ce..9bca7a7 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -63,6 +63,15 @@ services: - dspace__P__server__P__url=http://backend:8080/server # dspace.ui.url is used by the backend to generate frontend links (e.g. e-mail alerts). - dspace__P__ui__P__url=http://localhost:4000 + # Disable OIDC authentication for local dev; use password auth only. + # In production/staging this is set via the Kubernetes ConfigMap (backend-cm.jsonnet). + - plugin__P__sequence__P__org__P__dspace__P__authenticate__P__AuthenticationMethod=org.dspace.authenticate.PasswordAuthentication + # OidcAuthenticationBean is instantiated by Spring even when removed from the plugin + # sequence above. It calls String.split() on ip.bioIPsRange1/2 unconditionally, so + # provide harmless non-routable CIDR placeholders to prevent NullPointerException. + # In production/staging real IP ranges are supplied via the Kubernetes ConfigMap. + - ip__P__bioIPsRange1=192.0.2.0/24 + - ip__P__bioIPsRange2=192.0.2.0/24 ports: - "8080:8080" volumes: diff --git a/dspace/README.md b/dspace/README.md index 68067df..be2efa4 100644 --- a/dspace/README.md +++ b/dspace/README.md @@ -10,7 +10,7 @@ Run the GitHub action workflows in the following order to build the images that | [2. Build dspace images](https://github.com/mlibrary/dspace-containerization/actions/workflows/build-dspace-images.yml) | dspace-frontend:`tag`, dspace-backend:`tag`, dspace-solr:`tag`, dspace-db:`tag` | NOTE: -* Production configuration is **not** baked into the backend image. It is delivered at runtime via Kubernetes Secrets mounted directly as single files into the container (e.g. `dspace.cfg`, `authentication-oidc.cfg`). The actual Secret names and mount paths are managed in [deepblue-documents-kube](https://github.com/mlibrary/deepblue-documents-kube). +* Production configuration is **not** baked into the backend image. It is supplied at runtime entirely via environment variables — non-sensitive settings (DSpace property overrides, IP ranges, mail settings, etc.) from a Kubernetes **ConfigMap**, and sensitive values (credentials, API keys) from Kubernetes **Secrets**. Both are managed in [deepblue-documents-kube](https://github.com/mlibrary/deepblue-documents-kube). This mirrors the pattern used locally: the `backend` service in `docker-compose.yml` sets the same `__P__`-encoded env vars for local development. ## configuration and deployment Argo CD is used for configuration and deployment via [deepblue-documents-kube](https://github.com/mlibrary/deepblue-documents-kube) repository. From a70f2cab0f3ee84e6db8479bfdc1c100b073fef8 Mon Sep 17 00:00:00 2001 From: Greg Kostin Date: Mon, 27 Apr 2026 20:26:53 -0400 Subject: [PATCH 2/3] Close out: Replace local.cfg and Update Markdown Files tasks --- DONE.md | 27 +++++++++++++++++++++++++++ TODO.md | 22 ---------------------- 2 files changed, 27 insertions(+), 22 deletions(-) diff --git a/DONE.md b/DONE.md index 54566ad..9152f09 100644 --- a/DONE.md +++ b/DONE.md @@ -1,5 +1,32 @@ # DONE +## 2026-04-27T00:00:00 — Update Markdown Files for local.cfg Removal +Updated all markdown files to reflect the env-var-based configuration approach. +`dspace/README.md` NOTE corrected from "Kubernetes Secrets mounted as files" to +the accurate ConfigMap/env-var description. Confirmed no other markdown files +had stale `local.cfg` or `backend/config/` references. All tables validated +with `check_tables.py`. + +- [x] Update `dspace/README.md` NOTE: replace "Kubernetes Secrets mounted as files" with the correct ConfigMap/env-var description +- [x] Verify no other markdown files have stale references +- [x] Run `python3 dotpy/check_tables.py` on edited files +- [x] Verify with the developer that the task is complete + +## 2026-04-27T00:00:00 — Replace local.cfg with Docker Compose Environment Variables +Deleted `backend/local.cfg` and removed its `COPY` instruction from +`backend.dockerfile`. The three settings it contained — password-auth plugin +sequence, `ip.bioIPsRange1`, and `ip.bioIPsRange2` — are now expressed as +`__P__`-encoded `environment:` variables in the `backend` service of +`docker-compose.yml`, mirroring the Kubernetes ConfigMap pattern used in +production (`backend-cm.jsonnet`). `README.md` updated to remove all +`local.cfg` references and document the env-var approach. + +- [x] Add `plugin__P__sequence__P__org__P__dspace__P__authenticate__P__AuthenticationMethod` and IP-range placeholder env vars to the `backend` service in `docker-compose.yml` +- [x] Remove `COPY ./backend/local.cfg $DSPACE_INSTALL/config/local.cfg` from `backend.dockerfile` +- [x] Delete `backend/local.cfg` +- [x] Verify the current state of the project achieves the task goal +- [x] Verify with the developer that the task is complete + ## 2026-04-22T00:00:00 — Backend Bin Scripts README Documented all 58 scripts and data files in `backend/bin/` (excluding `README.md` itself and the CSV inventory sheets under `sheets/`). Initial diff --git a/TODO.md b/TODO.md index b49c5c5..539c82f 100644 --- a/TODO.md +++ b/TODO.md @@ -1,27 +1,5 @@ # TODO -## Replace local.cfg with Docker Compose Environment Variables -Remove `backend/local.cfg` and its `COPY` instruction from `backend.dockerfile`; -express the same settings as `environment:` entries in the `backend` service of -`docker-compose.yml`, mirroring the pattern used by the Kubernetes ConfigMap at -`deepblue-documents-kube/environments/deepblue-documents/demo/backend-cm.jsonnet`. - -- [x] Add `plugin__P__sequence__P__org__P__dspace__P__authenticate__P__AuthenticationMethod` and IP-range placeholder env vars to the `backend` service in `docker-compose.yml` -- [x] Remove `COPY ./backend/local.cfg $DSPACE_INSTALL/config/local.cfg` from `backend.dockerfile` -- [x] Delete `backend/local.cfg` -- [x] Verify the current state of the project achieves the task goal -- [ ] Verify with the developer that the task is complete - - - -## Update Markdown Files for local.cfg Removal -Update all markdown files to reflect the current project state after removing -`backend/local.cfg` and switching to env-var-based configuration. - -- [x] Update `dspace/README.md` NOTE: replace "Kubernetes Secrets mounted as files" with the correct ConfigMap/env-var description -- [x] Verify no other markdown files have stale references -- [x] Run `python3 dotpy/check_tables.py` on edited files -- [ ] Verify with the developer that the task is complete ## Scrub Deleted `.cpt` Files from Git History The five encrypted config files (`backend/config/*.cpt`) and the production log From e92780d9abf993df91739c8851d6068280183fb7 Mon Sep 17 00:00:00 2001 From: Greg Kostin Date: Mon, 27 Apr 2026 21:13:20 -0400 Subject: [PATCH 3/3] Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docker-compose.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker-compose.yml b/docker-compose.yml index 9bca7a7..22b7569 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -64,7 +64,7 @@ services: # dspace.ui.url is used by the backend to generate frontend links (e.g. e-mail alerts). - dspace__P__ui__P__url=http://localhost:4000 # Disable OIDC authentication for local dev; use password auth only. - # In production/staging this is set via the Kubernetes ConfigMap (backend-cm.jsonnet). + # In production/staging this is set via the external Kubernetes ConfigMap. - plugin__P__sequence__P__org__P__dspace__P__authenticate__P__AuthenticationMethod=org.dspace.authenticate.PasswordAuthentication # OidcAuthenticationBean is instantiated by Spring even when removed from the plugin # sequence above. It calls String.split() on ip.bioIPsRange1/2 unconditionally, so