feat: run single integration tests from dev mode (#14486)

* feat: allow running single integration tests from dev mode

Add integration-tests service with a Docker Compose profile to the dev
override, so a single integration test can be run against the running
dev stack without switching environments via setEnv.sh.

Also ignore the harmless Cross-Origin-Opener-Policy Chrome warning in
integration tests (occurs over HTTP) and remove the now-unnecessary
DD_SECURE_CROSS_ORIGIN_OPENER_POLICY overrides. Configure logging in
the integration test base class so test output is visible.

* fix: keep DD_SECURE_CROSS_ORIGIN_OPENER_POLICY in integration tests override

The base_test_class.py change to ignore the COOP warning is not yet on
the bugfix branch, so the full integration test suite still needs this
setting to avoid Chrome SEVERE console errors over HTTP.

* fix: add executable bit and platform: linux/amd64 for integration-tests

- Set executable bit on run-integration-test-dev.sh
- Add platform: "linux/amd64" to integration-tests service in both dev
  and integration_tests overrides (Chromium doesn't work on aarch64)

Author: valentijnscholten

docker-compose.override.dev.yml

@@ -73,3 +73,22 @@ services:
         mode: host
   "webhook.endpoint":
     image: mccutchen/go-httpbin:2.20.0@sha256:b1620821b6ff191d911629f87a720b88df5397c2554045f1cfb1ffde17c9b898
+  integration-tests:
+    platform: "linux/amd64"
+    profiles:
+      - integration
+    build:
+      context: ./
+      dockerfile: ${INTEGRATION_TESTS_DOCKERFILE:-Dockerfile.integration-tests-debian}
+    image: "defectdojo/defectdojo-integration-tests:${INTEGRATION_TESTS_VERSION:-latest}"
+    depends_on:
+      - nginx
+      - uwsgi
+    entrypoint: ['/wait-for-it.sh', '${DD_DATABASE_HOST:-postgres}:${DD_DATABASE_PORT:-5432}', '-t', '30', '--', '/app/docker/entrypoint-integration-tests.sh']
+    volumes:
+      - '.:/app:z'
+    environment:
+      DD_BASE_URL: 'http://nginx:8080/'
+      DD_ADMIN_USER: "${DD_ADMIN_USER:-admin}"
+      DD_ADMIN_PASSWORD: "${DD_ADMIN_PASSWORD:-admin}"
+      DD_INTEGRATION_TEST_FILENAME: "${DD_INTEGRATION_TEST_FILENAME:-}"

docker-compose.override.integration_tests.yml

@@ -1,6 +1,7 @@
 ---
 services:
   integration-tests:
+    platform: "linux/amd64"
     build:
       context: ./
       dockerfile: ${INTEGRATION_TESTS_DOCKERFILE:-Dockerfile.integration-tests-debian}

readme-docs/DOCKER.md

@@ -353,6 +353,20 @@ Check the logs with:
 docker compose logs -f integration-tests
 ```
 
+### Running a single integration test from dev mode
+
+If your dev stack is already running (`docker/setEnv.sh dev && docker compose up`), you can run a single integration test without switching environments.
+The dev override includes an `integration-tests` service behind a Docker Compose profile, so it won't start during normal `docker compose up`.
+
+Make sure the dev containers are up and healthy before running the test:
+
+```
+./run-integration-test-dev.sh tests/finding_test.py
+```
+
+This spins up the integration test runner container against your running dev stack and tears it down when done.
+Note that test data is created in your dev database — the integration tests attempt to clean up after themselves.
+
 # Checking Docker versions
 
 Run the following to determine the versions for docker and docker compose:

run-integration-test-dev.sh

@@ -0,0 +1,16 @@
+#!/bin/bash
+# Run a single integration test against the running dev stack.
+# Requires dev mode to be active (./docker/setEnv.sh dev && docker compose up).
+#
+# Usage:
+#   ./run-integration-test-dev.sh tests/finding_test.py
+#   ./run-integration-test-dev.sh tests/risk_acceptance_test.py
+
+set -e
+
+TEST_FILE="${1:?Usage: $0 <test-file>}"
+
+docker compose --profile integration run --rm --no-deps \
+  -e "DD_INTEGRATION_TEST_FILENAME=${TEST_FILE}" \
+  -e "LOG_LEVEL=${LOG_LEVEL:-INFO}" \
+  integration-tests

tests/base_test_class.py

@@ -13,6 +13,10 @@
 from selenium.webdriver.support.ui import WebDriverWait
 
 # import time
+logging.basicConfig(
+    level=os.environ.get("LOG_LEVEL", "WARNING"),
+    format="%(levelname)s %(name)s: %(message)s",
+)
 logger = logging.getLogger(__name__)
 
 
@@ -403,7 +407,7 @@ def assertNoConsoleErrors(self):
             The addition of the trigger exception is due to the Report Builder tests.
             The addition of the innerHTML exception is due to the test for quick reports in finding_test.py
             """
-            accepted_javascript_messages = r"(zoom\-in\.cur.*)404\ \(Not\ Found\)|Uncaught TypeError: Cannot read properties of null \(reading \'trigger\'\)|Uncaught TypeError: Cannot read properties of null \(reading \'innerHTML\'\)"
+            accepted_javascript_messages = r"(zoom\-in\.cur.*)404\ \(Not\ Found\)|Uncaught TypeError: Cannot read properties of null \(reading \'trigger\'\)|Uncaught TypeError: Cannot read properties of null \(reading \'innerHTML\'\)|Cross-Origin-Opener-Policy header has been ignored"
 
             if entry["level"] == "SEVERE":
                 # TODO: actually this seems to be the previous url
@@ -420,12 +424,12 @@ def assertNoConsoleErrors(self):
                     + self.driver.current_url,
                 )
                 if self.accept_javascript_errors:
-                    logger.warning(
+                    logger.debug(
                         "skipping SEVERE javascript error because accept_javascript_errors is True!",
                     )
                 elif re.search(accepted_javascript_messages, entry["message"]):
-                    logger.warning(
-                        "skipping javascript errors related to known issues images, see https://github.com/DefectDojo/django-DefectDojo/blob/master/tests/base_test_class.py#L324",
+                    logger.debug(
+                        "skipping javascript errors related to known issues, see https://github.com/DefectDojo/django-DefectDojo/blob/master/tests/base_test_class.py#L324",
                     )
                 else:
                     self.assertNotEqual(entry["level"], "SEVERE")