fix!: avoid git submodule init hook merges

.envrc

@@ -0,0 +1,9 @@
+#!/bin/bash
+
+# Automatically sets up your devbox environment whenever you cd into this
+# directory via our direnv integration:
+
+eval "$(devbox generate direnv --print-envrc)"
+
+# check out https://www.jetpack.io/devbox/docs/ide_configuration/direnv/
+# for more details

.github/workflows/test.yml

@@ -0,0 +1,31 @@
+# This workflow runs the BATS test suite on every pull request and every push to master.
+# Tests are co-located with the plugin they cover:
+#   devbox-plugins/base-config/tests/
+#
+# bats-support and bats-assert are registered as git submodules pinned to specific
+# tag SHAs and are populated here via actions/checkout submodules: recursive.
+on:
+  pull_request:
+  push:
+    branches:
+      - master
+
+name: test
+
+jobs:
+  test-plugin-base-config:
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        os: [ubuntu-latest, macos-latest]
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          # Populates bats-support and bats-assert lib submodules
+          submodules: recursive
+
+      - name: Install devbox
+        uses: jetify-com/devbox-install-action@v0.15.0
+
+      - name: Run base-config plugin tests
+        run: devbox run test

.gitmodules

@@ -0,0 +1,6 @@
+[submodule "devbox-plugins/base-config/tests/libs/bats-support"]
+	path = devbox-plugins/base-config/tests/libs/bats-support
+	url = https://github.com/bats-core/bats-support
+[submodule "devbox-plugins/base-config/tests/libs/bats-assert"]
+	path = devbox-plugins/base-config/tests/libs/bats-assert
+	url = https://github.com/bats-core/bats-assert

README.md

@@ -26,18 +26,19 @@ Example:
 
 | Plugin | Description |
 | --- | --- |
-| [base-config](#base-config) | Base configuration for a DevOps project - our initial proof-of-concept with everything. |
+| [base-config](#base-config) | Base configuration for a DevOps project - our initial proof-of-concept with everything, that do not yet take "archetypes" into account and such. |
 
-### Base config
+### base-config
 
-This plugin has the baseline of tools needed for various DevOps projects.
+This plugin has the baseline of tools needed for our own various DevOps projects.
 
 This includes:
 
 * Python, and activation of a virtual environment if `requirements.txt` is present in the root of the repository using this plugin
 * Pre-commit and linting hooks, and CI Github actions for it.
 * tenv, tflint and terraform-docs for working with terraform or tofu
 * Taskfile or reusable tasks
+* _ensure-submodules-populated:_ Init hook ensure git submodules are initialized with content, to guard rail against other tools that do not automatically detect or warn about missing contents and acts as nothing is wrong. Only populates submodules marked `-` by `git submodule status` (uninitialized). Submodules with ` ` (up-to-date), `+` (different SHA / local commits), or `U` (merge conflict) prefixes are **never touched**. Skip projects without git submodules as fast as possible, by checking `.gitmodules` first (fast exit), then `git rev-parse`.
 
 Assumptions:
 
@@ -73,7 +74,7 @@ Using 'tenv' you need to install either, the selection of the correct one, isn't
 
 #### Scripts
 
-The base config plugin includes some scripts to help bootstrap a new project.
+The `base-config` plugin includes some devbox scripts to help bootstrap a new project.
 
 Available scripts:
 
@@ -98,9 +99,10 @@ The following credentials are **automatically provisioned** via Terraform in the
 | Secret | `APP_OC_GITHUB_ACTIONS_PEM` | The GitHub App private key (PEM format) |
 
 **How it works:**
-- Repositories are configured in `github-infrastructure` Terraform with `add_app_oc_github_actions = true`
-- Terraform automatically provisions these credentials to the repository's Actions secrets/variables
-- The GitHub App itself is created once; credentials are then distributed to repos via Terraform
+
+* Repositories are configured in `github-infrastructure` Terraform with `add_app_oc_github_actions = true`
+* Terraform automatically provisions these credentials to the repository's Actions secrets/variables
+* The GitHub App itself is created once; credentials are then distributed to repos via Terraform
 
 If a repository is missing these credentials, it needs to be configured in the `github-infrastructure` repository's Terraform configuration.
 
@@ -110,7 +112,7 @@ The devbox script common `bootstrap-taskfile` will create a taskfile in the root
 
 ## Roadmap
 
-* If needed we could split the base config into optional plugins eg. a python plugin, a terraform plugin etc.
+* If needed we could split the `base-config` plugin into optional "archetypes" kind of plugins eg. a python plugin, a terraform plugin etc.
 * Make bootstrap scripts idempotent (seen from the consuming projects point of view) or consider the bootstrap vs configuration concept described below.
 * Notice the devbox have a nice feature of writing a project readme. We don't use it yet, but could create a 2nd readme in our projects and under our project specific tooling and usage sections we always have in our project just link to this 2nd readme that is automatically generated and be kept updated automatically as well. Only problem if the autogenerated readme contain user specific paths, but it might be possible to clean up with `sed` or just wait for devbox to improve.
 
@@ -131,3 +133,27 @@ As a first effort of trying to keep projects using the taskfile and devbox plugi
 Additions are typically not breaking, but moving configuration to other files or removing configuration might be breaking and thus should be cleaned up.
 
 We supply devbox run commands for migrations, e.g. `devbox run migration-1x-to-2x` for migrating from version 1.x to 2.x. The command or helper scripts document the details.
+
+## Tests
+
+As part of a bugfix, a proof-of-concept for testing the devbox plugin functionality was created. The tests verifies functionality related to the changes, and is planned to serve as inspiration for further tests.
+
+We have chosen to use [BATS](https://bats-core.readthedocs.io/en/stable/) ([source code](https://github.com/bats-core)) for testing, in the `base-config` plugin as it seems a reasonable fit testing shell related functionality as a devbox plugin, and especially the shell scripts used.
+
+For now the tests are located in the `tests` folder of the plugin. There is currently only one devbox plugin in this repository, no effort of re-use across multiple plugins with BATS helper functions have been done, nor is any wrapper functionality created in the root of the repository to collect all future tests (devbox can easily be used though).
+
+See details about specific tests in the READMEs in the `tests` folders. First one in [`base-config` plugin testing](devbox-plugins/base-config/tests/README.md).
+
+### BATS test framework guidelines
+
+* Use Git submodules to include BATS helper libraries like their [`bats-assert`](https://github.com/bats-core/bats-assert), so we can security review the code and ensure it is locked dependency we fully manage.
+* BATS are installed through devbox in the is project.
+* Use a devbox run script `test` to run the tests, easily usable in CI actions and locally with `devbox run test`.
+
+## Releases
+
+See [CHANGELOG.md](CHANGELOG.md) for releases, migration and compatibility between versions.
+
+Release process is described here: [https://docs.devops.oc.dk/hww/release-process/](https://docs.devops.oc.dk/hww/release-process/)
+
+_Sorry for the internal links in a public repository, but you basically just need to know it follows [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/#summary) and we use [release-please](https://github.com/googleapis/release-please) to automate the release process_.

devbox-plugins/base-config/config/ensure-submodules-populated.sh

@@ -0,0 +1,49 @@
+#!/usr/bin/env bash
+# Ensures all git submodules have content (are populated), without modifying
+# submodules that are already initialized -- even if they are at a different
+# SHA or have local changes.
+#
+# Only submodules marked with '-' by `git submodule status` (uninitialized /
+# empty) are populated.  Submodules marked ' ' (up to date), '+' (different
+# SHA / local commits) or 'U' (merge conflict) are left completely untouched.
+#
+# This is intentionally non-intrusive: it is safe to run on every devbox shell
+# enter without risking loss of in-progress work inside a submodule.
+
+set -euo pipefail
+
+
+# Guard order matters: checks `.gitmodules` first (fast exit), then `git rev-parse`
+# to see if the repository is a git repository as all.
+# Gracefully skip when there is nothing to do
+if [ ! -f .gitmodules ]; then
+    echo "ensure-submodules-populated: git submodules not detected, skipping"
+    exit 0
+fi
+
+if ! git rev-parse --git-dir > /dev/null 2>&1; then
+    echo "ensure-submodules-populated: not inside a git repository, skipping"
+    exit 0
+fi
+
+# Collect the paths of uninitialized submodules only.
+# `git submodule status` output format:  <prefix><sha> <path> [<description>]
+# prefix '-' means the submodule has never been initialized (directory is empty).
+mapfile -t uninitialized < <(git submodule status | awk '$0 ~ /^-/ { print $2 }')
+
+if [ "${#uninitialized[@]}" -eq 0 ]; then
+    echo "ensure-submodules-populated: all git submodules are already initialized, skipping"
+    exit 0
+fi
+
+echo "ensure-submodules-populated: found ${#uninitialized[@]} uninitialized submodule(s), populating..."
+
+for path in "${uninitialized[@]}"; do
+    echo "  populating submodule: ${path}"
+    # --init      : register the submodule if not yet in .git/config
+    # --recursive : also populate nested submodules inside this one
+    # --           : explicit end-of-options to safely handle unusual paths
+    git submodule update --init --recursive -- "${path}"
+done
+
+echo "ensure-submodules-populated: done"

devbox-plugins/base-config/plugin.json

@@ -70,7 +70,8 @@
     "{{ .Virtenv }}/devbox-requirements.txt": "config/devbox-requirements.txt",
     "{{ .Virtenv }}/migrate-1x-to-2x-helper.sh": "config/migrate-1x-to-2x-helper.sh",
     "{{ .Virtenv }}/alias-terraform-tofu-init.sh": "config/alias-terraform-tofu-init.sh",
-    "{{ .Virtenv }}/init-submodules.sh": "config/init-submodules.sh"
+    "{{ .Virtenv }}/init-submodules.sh": "config/init-submodules.sh",
+    "{{ .Virtenv }}/ensure-submodules-populated.sh": "config/ensure-submodules-populated.sh"
   },
   "shell": {
     "init_hook": [
@@ -103,8 +104,11 @@
       "if crudini --get .gitignore \"\" \".aliases\";then true; else echo -e '\n\n# Aliases dir with script for devbox\n.aliases' >> .gitignore; fi",
       // run the different alias scripts:
       "bash {{ .Virtenv }}/alias-terraform-tofu-init.sh",
-      // ensure git submodules are initialized and updated so we don't have empty submodule dirs (exits clean if no submodules are present) - also merge any local commits to the update as they would else be detached
-      "git submodule update --init --recursive --merge"
+      // Ensure all git submodules have content (are populated). Only uninitialized (empty) submodules are
+      // populated; submodules that already have content -- including ones with local commits or uncommitted
+      // changes -- are left completely untouched. This prevents silent failures where tools like Ansible
+      // silently operate on an empty submodule directory.
+      "bash {{ .Virtenv }}/ensure-submodules-populated.sh"
     ],
     "scripts": {
       "bootstrap-envrc-private": [

devbox-plugins/base-config/tests/README.md

@@ -0,0 +1,9 @@
+# devbox base-config plugin tests
+
+This is just a proof-of-concept, as mentioned the [repository root README](../../../README.md).
+
+Overview of tests, possible further test specific details and implementation details in subsections below.
+
+| Test name | Description |
+| --- | --- |
+| `ensure-submodules-populated.bats` | Verifies the shell script that is called in the plugin init hook, to detect uninitialized git submodules of a project and initializes the with content, to guard rail against other tools that do not automatically detect or warn about missing contents and acts as nothing is wrong (_Looking angry a you Ansible!_). |