Add a Zuul image content provider by SeanMooney · Pull Request #66 · openstack-k8s-operators/s2i-openstack-containers

SeanMooney · 2026-08-12T16:51:37Z

Summary

add shared image content-provider playbooks and a Zuul adapter
publish selected images to the buildset registry and return exact references
validate central deployment mappings, registry state, returned data, and exact cleanup
add ordered target unions, machine-readable target/reference output, authenticated pushes, and live parallel logs
add the Molecule provider contract and attach both provider jobs to GitHub check

Validation

uvx --python 3.13 tox -e unit (43 passed)
commit-time pre-commit checks
git diff --check

Dependencies

Depends on #63, which in turn depends on replacement PR #80. This PR targets main directly and temporarily shows prerequisite commits until they merge.

PRs #64, #65, and #79 are already merged into main.

This change intentionally excludes third-party static-node reset support, speculative service checkouts, and operator builds.

Introduce two consumer-side additions to the content provider pipeline: 1. A non-voting Zuul job (s2i-openstack-container-consumer-smoke) that depends on the content provider, pulls all published images on a fresh consumer node, inspects them, validates deployment key mappings, and smoke-tests Watcher service entrypoints. 2. A unit test (test_deployment_keys.py) that validates every deployment key declared in containers/image-mappings.yaml against the OpenStackVersion.spec.customContainerImages fields from openstack-operator. Covers ContainerTemplate and ContainerDefaults structs. When a local openstack-operator checkout is available the test parses the Go source live; otherwise it falls back to a frozen field set and flags staleness. Both additions strengthen the contract between s2i-built images and the OpenStack deployment machinery without requiring a cluster. Depends-On: openstack-k8s-operators#66 Assisted-By: Claude (Anthropic) Co-authored-by: Cursor <cursoragent@cursor.com>

Add two non-voting jobs that validate the openstack-tempest container image built by s2i-openstack-containers: 1. KUTTL job (test-operator-kuttl-s2i-tempest): patches the operator deployment to default to the s2i image, then runs the tempest-s2i KUTTL suite validating that the Tempest CR reconciles to Ready. 2. E2E job (podified-multinode-edpm-deployment-crc-test-operator-s2i-tempest): full CRC deployment using the s2i-built tempest image to run actual tempest tests against a live OpenStack environment. Both jobs depend on s2i-openstack-container-content-provider to build the tempest image from source via the buildset registry. Depends-On: openstack-k8s-operators/s2i-openstack-containers#26 Depends-On: openstack-k8s-operators/s2i-openstack-containers#66 Co-authored-by: Cursor <cursoragent@cursor.com>

rebtoor · 2026-08-14T09:42:22Z

+        mode: "0600"
+        content: |
+          {{ {
+               'container': ('buildset_registry' if s2i_ci_registry_port | int == 5000 else 'buildset_registry_' + s2i_ci_registry_port | string),


The expression:

'buildset_registry' if s2i_ci_registry_port | int == 5000 else 'buildset_registry_' + s2i_ci_registry_port | string

is repeated 4 times across pre.yaml and post.yaml. If the naming convention changes, all sites must be updated in lockstep.

Consider computing it once as a fact early in pre.yaml:

- name: Compute the buildset registry container name ansible.builtin.set_fact: s2i_ci_registry_container_name: >- {{ 'buildset_registry' if s2i_ci_registry_port | int == 5000 else 'buildset_registry_' + s2i_ci_registry_port | string }}

Then reference s2i_ci_registry_container_name everywhere, and store it in the ownership marker. In post.yaml, read it from the marker (s2i_ci_registry_owner.container), which you already do in some places but not all.

rebtoor · 2026-08-14T09:55:58Z

+          - "{{ s2i_ci_registry_image }}"
+          - zuul-registry
+          - -d
+      changed_when: true


nit: This sequence (start via run-buildset-registry role, then immediately destroy and recreate with :Z labels if SELinux is enabled) is a workaround for the upstream role not supporting SELinux volume labels.

Please add a comment explaining why this is necessary:

# Workaround: the run-buildset-registry role from zuul-jobs does not # mount volumes with :Z labels. On SELinux-enforcing nodes the registry # cannot access its TLS/conf directories. Remove and recreate with # the correct labels. Track upstream fix: <link if applicable>.

Without this, a future maintainer may remove the "redundant" restart thinking it's a bug.

rebtoor · 2026-08-14T10:00:46Z

+    def test_parallel_output_is_live_and_logs_are_retained(self):
+        process = subprocess.Popen(
+            [
+                str(self.root / "build.sh"),
+                "build-parallel",
+                "alpha/one,beta/two",
+            ],
+            cwd=self.root,
+            env=self._environment(),
+            stdout=subprocess.PIPE,
+            stderr=subprocess.STDOUT,
+            text=True,
+        )
+        self.addCleanup(
+            lambda: process.kill() if process.poll() is None else None
+        )
+        self.addCleanup(process.stdout.close)
+        output = []
+        deadline = time.monotonic() + 5
+        while time.monotonic() < deadline:
+            line = process.stdout.readline()
+            output.append(line)
+            if "LIVE" in line:
+                break
+
+        self.assertIn("LIVE", "".join(output))
+        self.assertIsNone(
+            process.poll(), "build exited before live output arrived"
+        )
+        output.append(process.stdout.read())


The test relies on a 750ms time.sleep() in the fake buildah script to validate that live output appears while the process is still running. Under CI load, the 5-second deadline may not be sufficient if the scheduler starves the subprocess.

Consider either:

Increasing the deadline generously (e.g., 30s) since you're only waiting for the first "LIVE" line
Using a synchronization mechanism (e.g., a fifo/pipe that the fake script writes to and the test blocks on)
At minimum, adding a retry or a comment acknowledging the timing sensitivity
If flaky test reports appear, this is the likely culprit.

rebtoor · 2026-08-14T10:02:34Z

+        custom_container_images: "{{ s2i_ci_custom_container_images }}"
+      content_provider_os_custom_container_images: >-
+        {{ s2i_ci_custom_container_images }}
+      content_provider_os_registry_url: "null"


This sets a string "null", not YAML null. In Ansible, "null" | bool evaluates to False (which happens to be correct), but "null" | length > 0 is True, making it look non-empty.

A brief inline comment documenting how consumers should check this would prevent misuse:

# Intentional string sentinel -- consumers test `!= "null"` or check # content_provider_registry_available instead. Cannot be actual null # because Zuul zuul_return merges dicts and null would not override # a parent job's existing value. content_provider_os_registry_url: "null"

Also worth mentioning in the developer guide's "Registry and returned data" section.

rebtoor · 2026-08-14T10:04:11Z

+    - name: Stop the project-owned buildset registry tunnel
+      when:
+        - s2i_ci_registry_owner_loaded | bool
+        - s2i_ci_registry_owner_is_valid | bool
+      ansible.builtin.shell: |
+        pids=$(pgrep -f '^socat -d -d TCP6-LISTEN:{{ s2i_ci_registry_owner.port }},fork TCP:127.0.0.1:1{{ s2i_ci_registry_owner.port }}$' || true)
+        if [ -n "$pids" ]; then
+          printf '%s\n' "$pids"
+          kill $pids
+        fi
+      args:
+        executable: /bin/bash
+      register: s2i_ci_stopped_registry_tunnel
+      changed_when: s2i_ci_stopped_registry_tunnel.stdout | length > 0


The pgrep -f pattern:

'^socat -d -d TCP6-LISTEN:{{ port }},fork TCP:127.0.0.1:1{{ port }}$'

is fragile — if the socat invocation changes (flag order, IPv4 vs IPv6, verbosity level), the pattern won't match and the tunnel leaks silently.

A more robust approach: record the socat PID in the ownership marker during pre.yaml, then kill it by PID in post.yaml:

# In pre.yaml, after starting socat: s2i_ci_registry_owner: container: buildset_registry port: 5000 tunnel_pid: "{{ socat_pid }}"

This makes cleanup deterministic regardless of command-line format changes.

rebtoor · 2026-08-14T10:04:59Z

  - libxslt-devel
  - mod_ssl
  - openssl-devel
+  - openssl-libs


Nit: does this belong to this PR?

rebtoor · 2026-08-14T10:05:03Z

 libffi
 libxml2
 libxslt
+openssl-libs


Nit: does this belong to this PR?

Actually, i think it's not needed, as it transitive requirements of others. Also, tbh openssl-lib is so basic that is installed by default even in the ubi minimal. I'd keep it out of this one.

rebtoor · 2026-08-14T10:07:04Z

+    - name: Validate consolidated Watcher process entry points
+      ansible.builtin.command:
+        argv:
+          - podman
+          - run
+          - --rm
+          - --user
+          - watcher
+          - --entrypoint
+          - /bin/sh
+          - "{{ s2i_ci_local_target_references['watcher/watcher-base'] }}"
+          - -ec
+          - >-
+            for command in watcher-api watcher-applier watcher-decision-engine;
+            do command -v "$command"; "$command" --help >/dev/null; done
+      when: "'watcher/watcher-base' in s2i_ci_selected_images"
+      changed_when: false


This task couples the shared build/publish pipeline to one specific image's runtime expectations. As more images land (tempest, glance, manila), each would need a similar hardcoded block here.

Consider making this pluggable, for example, if containers/<image>/validate.sh exists, the provider runs it post-build. That keeps image-specific knowledge inside each image directory and removes the need to modify the shared pipeline for every new image.

At minimum, a comment explaining this is intentional and what the long-term plan is would help.

ya i will remove this that was from before i had zuul working.

i start with a purly local version of this and the wrote the zuul job last
and fliped the order when submitting. to have the zuul job first but i do not need this validation anymroe

this woudl be more appreate in say the molecuel tests rather thne in the main shared run palybook but ill drop it for now

amoralej

I focused my review the build.sh part. I like most of the changes, all but:

I don't like automatically adding base container, see my suggestion about setting BASE_IMAGE.
My concern about moving tests to python as expressed in #63

amoralej · 2026-08-14T10:06:20Z

 DEFAULT_STREAM="${DEFAULT_STREAM:-master}"
 SKIP_HASH_UPDATE="${SKIP_HASH_UPDATE:-}"
 PIP_NO_BINARY="${PIP_NO_BINARY:-}"
+REGISTRY_AUTH_FILE="${REGISTRY_AUTH_FILE:-}"


I think REGISTRY_AUTH_FILE variable is used by default by buildah, but as we are adding REGISTRY_CERT_DIR option which is not, I'm fine with adding.

amoralej · 2026-08-14T10:09:59Z

    ;;
  build-parallel)
    _bp_targets=($(resolve_targets "${TARGETS[@]}"))
+    if [[ ! "${PARALLEL}" =~ ^[1-9][0-9]*$ ]]; then


amoralej · 2026-08-14T10:11:59Z

    for _bp_img in "${_bp_targets[@]}"; do
      [[ -n "$(project_name "${_bp_img}")" ]] && continue
-      build_image "${_bp_img}"
+      set -o pipefail


amoralej · 2026-08-14T10:29:37Z

 }

-# Resolve which images to process (accepts one or more targets)
+# Resolve one or more image expressions in their requested order. A


IIUC this is adding support to resolve comma separated list of targets in addition to current space separated lists.

About automatically adding base, i'm not sure we want that, new services image may (or should) just rely in the last version pushed to the public registry. quay.io/openstack-k8s-operators/openstack-base:master-testing . Actually i had been thinking if we should set BASE_IMAGE to that by default.

amoralej · 2026-08-14T10:30:48Z

+      done
+      if [[ ${selected_service} -eq 1 && -z "${seen[base]:-}" ]]; then
+        resolved=("base" "${resolved[@]}")
+        seen[base]=1


See my previous comment about base, i'm not sure we should automatically add base. In most cases i think we should just set BASE_IMAGE to quay.io/openstack-k8s-operators/openstack-base:master-testing

It's also tricky that base is only added when resolving comma separated list, not in space separated list. Looks like a "hidden" feature.

$ bash ./build.sh resolve watcher glance watcher/watcher-base glance/glance-api $ bash ./build.sh resolve watcher,glance base watcher/watcher-base glance/glance-api

I'd remove the autoinjection of base.

amoralej · 2026-08-14T10:38:30Z

      push_image "${img}"
    done
    ;;
+  refs)


+1 to adding these two commands

amoralej · 2026-08-14T11:42:16Z

 libffi
 libxml2
 libxslt
+openssl-libs


Actually, i think it's not needed, as it transitive requirements of others. Also, tbh openssl-lib is so basic that is installed by default even in the ubi minimal. I'd keep it out of this one.

amoralej · 2026-08-14T11:47:53Z

-        # Wait for a slot if at the limit
        while [[ ${_bp_running} -ge ${PARALLEL} ]]; do
-          if ! wait -n; then
+          _bp_finished=""


I think all these changes improve the parallel build logging and failure handling. In my tests, when a container fails to build one container in parallel, other ongoing parallel builds still run in background while control is returned which is not perfect but i think it's the best we can do so far (we don't want to kill -9 i guess). Return codes are fine which is the most important.

amoralej · 2026-08-14T12:12:47Z

Wrong branch

openshift-ci · 2026-08-14T14:39:33Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from rebtoor. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci · 2026-08-14T14:39:40Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from rebtoor. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

SeanMooney · 2026-08-14T14:43:13Z

I focused my review the build.sh part. I like most of the changes, all but:

* I don't like automatically adding base container, see my suggestion about setting BASE_IMAGE.

* My concern about moving tests to python as expressed in [#63](https://github.com/openstack-k8s-operators/s2i-openstack-containers/pull/63#discussion_r3782736788)

ya i am fixing up the PRs first

in my poc i moved the impelmaiton to python as one of the firt thing i did then i reordered the pathces to bring this to the start
ill revert the test back to bash once i have fixed my other prs

Dependency markers are evaluated by the Python interpreter running the resolver. Allowing different Python minor versions and unpinned generator tools therefore produces host-dependent lock files and noisy generator metadata. Require Python 3.12 for source and lock generation, pin the resolver tools, and normalize annotations, headers, and index configuration from generated locks. Add a pinned-source reproducibility workflow and architecture checks, then regenerate the tracked lock files with the canonical environment. Assisted-By: Pi gpt-5.6-sol Signed-off-by: Sean Mooney <work@seanmooney.info> (cherry picked from commit d28af26)

Source maintenance currently resolves and applies repositories one at a time. A late failure can therefore leave tracked source pins partially updated, and a moving branch can provide different content during one invocation. Preflight every selected source record and freeze each effective commit before tracked mutation. Retain the fetched objects for the complete run, record their authority in a deterministic manifest, and install source updates atomically. Replace the shell test harness with the existing stdlib unittest coverage for failure isolation, stream safety, checkout ownership, and multi-target scope. Assisted-By: Pi gpt-5.6-sol Signed-off-by: Sean Mooney <work@seanmooney.info> (cherry picked from commit 0a14e6a)

Dependent jobs need exact references to service images built from the current container repository, but the repository has no reusable job that publishes a selected image set to the Zuul buildset registry. Add shared provider playbooks, a Zuul adapter, and contract validation for registry handling, deployment mappings, returned data, and exact cleanup. Extend the build interface with ordered target unions, machine-readable target and reference output, authenticated pushes, and live parallel logs. Attach the provider and Molecule contract jobs to the GitHub check pipeline. Assisted-By: Pi gpt-5.6-sol Signed-off-by: Sean Mooney <work@seanmooney.info> (cherry picked from commit 028d932)

openshift-ci · 2026-08-14T21:19:18Z

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

rebtoor · 2026-08-17T10:25:01Z

/hold

Rebase and rework the content provider from PR openstack-k8s-operators#66 onto the current main branch. The provider builds and publishes OpenStack service container images to a Zuul buildset registry so that dependent jobs can consume exact image references. Changes from the original PR openstack-k8s-operators#66: - Drop the "Make source updates atomic" commit (keep existing test_update_sources.sh per amoralej's feedback). - Drop the "Standardize lock generation on Python 3.12" commit (already merged as part of PR openstack-k8s-operators#80). - Remove auto-base insertion from resolve_targets() as requested by amoralej; the caller is now responsible for including base. - Drop openssl-libs from watcher (transitive, already in UBI). - Remove hardcoded watcher entrypoint validation from the shared run playbook (SeanMooney ack'd). Review feedback addressed: - Compute buildset registry container name as a fact once instead of repeating the expression 4 times (rebtoor). - Add SELinux workaround comment explaining the container remove/recreate sequence (rebtoor). - Record socat tunnel PID in the ownership marker and kill by PID in post-run instead of using fragile pgrep patterns (rebtoor). - Document the "null" string sentinel in content-provider-return explaining why it cannot be actual null (rebtoor). - Increase test_provider_shell.py parallel output deadline from 5s to 30s to avoid CI flakiness (rebtoor). Co-Authored-By: Sean Mooney <work@seanmooney.info> Assisted-By: Claude (Anthropic) Co-authored-by: Cursor <cursoragent@cursor.com>

Introduce two consumer-side additions to the content provider pipeline: 1. A non-voting Zuul job (s2i-openstack-container-consumer-smoke) that depends on the content provider, pulls all published images on a fresh consumer node, inspects them, validates deployment key mappings, and smoke-tests Watcher service entrypoints. 2. A unit test (test_deployment_keys.py) that validates every deployment key declared in containers/image-mappings.yaml against the OpenStackVersion.spec.customContainerImages fields from openstack-operator. Covers ContainerTemplate and ContainerDefaults structs. When a local openstack-operator checkout is available the test parses the Go source live; otherwise it falls back to a frozen field set and flags staleness. Both additions strengthen the contract between s2i-built images and the OpenStack deployment machinery without requiring a cluster. Depends-On: openstack-k8s-operators#66 Assisted-By: Claude (Anthropic) Co-authored-by: Cursor <cursoragent@cursor.com>

Rebase and rework the content provider from PR openstack-k8s-operators#66 onto the current main branch. The provider builds and publishes OpenStack service container images to a Zuul buildset registry so that dependent jobs can consume exact image references. Changes from the original PR openstack-k8s-operators#66: - Drop the "Make source updates atomic" commit (keep existing test_update_sources.sh per amoralej's feedback). - Drop the "Standardize lock generation on Python 3.12" commit (already merged as part of PR openstack-k8s-operators#80). - Remove auto-base insertion from resolve_targets() as requested by amoralej; the caller is now responsible for including base. - Drop openssl-libs from watcher (transitive, already in UBI). - Remove hardcoded watcher entrypoint validation from the shared run playbook (SeanMooney ack'd). Review feedback addressed: - Compute buildset registry container name as a fact once instead of repeating the expression 4 times (rebtoor). - Add SELinux workaround comment explaining the container remove/recreate sequence (rebtoor). - Record socat tunnel PID in the ownership marker and kill by PID in post-run instead of using fragile pgrep patterns (rebtoor). - Document the "null" string sentinel in content-provider-return explaining why it cannot be actual null (rebtoor). - Increase test_provider_shell.py parallel output deadline from 5s to 30s to avoid CI flakiness (rebtoor). Co-Authored-By: Sean Mooney <work@seanmooney.info> Assisted-By: Claude (Anthropic) Co-authored-by: Cursor <cursoragent@cursor.com>

Introduce two consumer-side additions to the content provider pipeline: 1. A non-voting Zuul job (s2i-openstack-container-consumer-smoke) that depends on the content provider, pulls all published images on a fresh consumer node, inspects them, validates deployment key mappings, and smoke-tests Watcher service entrypoints. 2. A unit test (test_deployment_keys.py) that validates every deployment key declared in containers/image-mappings.yaml against the OpenStackVersion.spec.customContainerImages fields from openstack-operator. Covers ContainerTemplate and ContainerDefaults structs. When a local openstack-operator checkout is available the test parses the Go source live; otherwise it falls back to a frozen field set and flags staleness. Both additions strengthen the contract between s2i-built images and the OpenStack deployment machinery without requiring a cluster. Depends-On: openstack-k8s-operators#66 Assisted-By: Claude (Anthropic) Co-authored-by: Cursor <cursoragent@cursor.com>

Add two non-voting jobs that validate the openstack-tempest container image built by s2i-openstack-containers: 1. KUTTL job (test-operator-kuttl-s2i-tempest): patches the operator deployment to default to the s2i image, then runs the tempest-s2i KUTTL suite validating that the Tempest CR reconciles to Ready. 2. E2E job (podified-multinode-edpm-deployment-crc-test-operator-s2i-tempest): full CRC deployment using the s2i-built tempest image to run actual tempest tests against a live OpenStack environment. Both jobs depend on s2i-openstack-container-content-provider to build the tempest image from source via the buildset registry. Depends-On: openstack-k8s-operators/s2i-openstack-containers#26 Depends-On: openstack-k8s-operators/s2i-openstack-containers#66 Co-authored-by: Cursor <cursoragent@cursor.com>

Rebase and rework the content provider from PR openstack-k8s-operators#66 onto the current main branch. The provider builds and publishes OpenStack service container images to a Zuul buildset registry so that dependent jobs can consume exact image references. Additionally, incorporates the speculative build support from PR openstack-k8s-operators#49 (reimplemented without OIB) to enable automatic detection of affected images and source staging for Zuul-driven speculative builds: - `build.sh auto-detect <project> [stream]` scans sources.txt files to find which images reference a given upstream project. - `build.sh list-sources <image> [stream]` lists source dependencies in a structured format for Ansible consumption. - `shared/resolve-auto-images.yaml` resolves `s2i_ci_images: auto` using the Zuul change queue. - `shared/stage-zuul-sources.yaml` copies Zuul checkouts into container build contexts for speculative builds. - `shared/run.yaml` is updated with hooks for auto-detection and source staging, invoked before input validation and build steps. Changes from the original PR openstack-k8s-operators#66: - Drop the "Make source updates atomic" commit (keep existing test_update_sources.sh per amoralej's feedback). - Drop the "Standardize lock generation on Python 3.12" commit (already merged as part of PR openstack-k8s-operators#80). - Remove auto-base insertion from resolve_targets() as requested by amoralej; the caller is now responsible for including base. - Drop openssl-libs from watcher (transitive, already in UBI). - Remove hardcoded watcher entrypoint validation from the shared run playbook (SeanMooney ack'd). Review feedback addressed: - Compute buildset registry container name as a fact once instead of repeating the expression 4 times (rebtoor). - Add SELinux workaround comment explaining the container remove/recreate sequence (rebtoor). - Record socat tunnel PID in the ownership marker and kill by PID in post-run instead of using fragile pgrep patterns (rebtoor). - Document the "null" string sentinel in content-provider-return explaining why it cannot be actual null (rebtoor). - Increase test_provider_shell.py parallel output deadline from 5s to 30s to avoid CI flakiness (rebtoor). Co-Authored-By: Sean Mooney <work@seanmooney.info> Assisted-By: Claude (Anthropic) Co-authored-by: Cursor <cursoragent@cursor.com>

Rebase and rework the content provider from PR openstack-k8s-operators#66 onto the current main branch. The provider builds and publishes OpenStack service container images to a Zuul buildset registry so that dependent jobs can consume exact image references. Changes from the original PR openstack-k8s-operators#66: - Drop the "Make source updates atomic" commit (keep existing test_update_sources.sh per amoralej's feedback). - Drop the "Standardize lock generation on Python 3.12" commit (already merged as part of PR openstack-k8s-operators#80). - Remove auto-base insertion from resolve_targets() as requested by amoralej; the caller is now responsible for including base. - Drop openssl-libs from watcher (transitive, already in UBI). - Remove hardcoded watcher entrypoint validation from the shared run playbook (SeanMooney ack'd). Review feedback addressed: - Compute buildset registry container name as a fact once instead of repeating the expression 4 times (rebtoor). - Add SELinux workaround comment explaining the container remove/recreate sequence (rebtoor). - Record socat tunnel PID in the ownership marker and kill by PID in post-run instead of using fragile pgrep patterns (rebtoor). - Document the "null" string sentinel in content-provider-return explaining why it cannot be actual null (rebtoor). - Increase test_provider_shell.py parallel output deadline from 5s to 30s to avoid CI flakiness (rebtoor). Co-Authored-By: Sean Mooney <work@seanmooney.info> Assisted-By: Claude (Anthropic) Co-authored-by: Cursor <cursoragent@cursor.com>

Introduce two consumer-side additions to the content provider pipeline: 1. A non-voting Zuul job (s2i-openstack-container-consumer-smoke) that depends on the content provider, pulls all published images on a fresh consumer node, inspects them, validates deployment key mappings, and smoke-tests Watcher service entrypoints. 2. A unit test (test_deployment_keys.py) that validates every deployment key declared in containers/image-mappings.yaml against the OpenStackVersion.spec.customContainerImages fields from openstack-operator. Covers ContainerTemplate and ContainerDefaults structs. When a local openstack-operator checkout is available the test parses the Go source live; otherwise it falls back to a frozen field set and flags staleness. Both additions strengthen the contract between s2i-built images and the OpenStack deployment machinery without requiring a cluster. Depends-On: openstack-k8s-operators#66 Assisted-By: Claude (Anthropic) Co-authored-by: Cursor <cursoragent@cursor.com>

Rebase and rework the content provider from PR openstack-k8s-operators#66 onto the current main branch. The provider builds and publishes OpenStack service container images to a Zuul buildset registry so that dependent jobs can consume exact image references. Changes from the original PR openstack-k8s-operators#66: - Drop the "Make source updates atomic" commit (keep existing test_update_sources.sh per amoralej's feedback). - Drop the "Standardize lock generation on Python 3.12" commit (already merged as part of PR openstack-k8s-operators#80). - Remove auto-base insertion from resolve_targets() as requested by amoralej; the caller is now responsible for including base. - Drop openssl-libs from watcher (transitive, already in UBI). - Remove hardcoded watcher entrypoint validation from the shared run playbook (SeanMooney ack'd). Review feedback addressed: - Compute buildset registry container name as a fact once instead of repeating the expression 4 times (rebtoor). - Add SELinux workaround comment explaining the container remove/recreate sequence (rebtoor). - Record socat tunnel PID in the ownership marker and kill by PID in post-run instead of using fragile pgrep patterns (rebtoor). - Document the "null" string sentinel in content-provider-return explaining why it cannot be actual null (rebtoor). - Increase test_provider_shell.py parallel output deadline from 5s to 30s to avoid CI flakiness (rebtoor). Co-Authored-By: Sean Mooney <work@seanmooney.info> Assisted-By: Claude (Anthropic) Co-authored-by: Cursor <cursoragent@cursor.com>

Introduce two consumer-side additions to the content provider pipeline: 1. A non-voting Zuul job (s2i-openstack-container-consumer-smoke) that depends on the content provider, pulls all published images on a fresh consumer node, inspects them, validates deployment key mappings, and smoke-tests Watcher service entrypoints. 2. A unit test (test_deployment_keys.py) that validates every deployment key declared in containers/image-mappings.yaml against the OpenStackVersion.spec.customContainerImages fields from openstack-operator. Covers ContainerTemplate and ContainerDefaults structs. When a local openstack-operator checkout is available the test parses the Go source live; otherwise it falls back to a frozen field set and flags staleness. Both additions strengthen the contract between s2i-built images and the OpenStack deployment machinery without requiring a cluster. Depends-On: openstack-k8s-operators#66 Assisted-By: Claude (Anthropic) Co-authored-by: Cursor <cursoragent@cursor.com>

SeanMooney · 2026-08-19T08:22:02Z

superceded by #85

Add two non-voting jobs that validate the openstack-tempest container image built by s2i-openstack-containers: 1. KUTTL job (test-operator-kuttl-s2i-tempest): patches the operator deployment to default to the s2i image, then runs the tempest-s2i KUTTL suite validating that the Tempest CR reconciles to Ready. 2. E2E job (podified-multinode-edpm-deployment-crc-test-operator-s2i-tempest): full CRC deployment using the s2i-built tempest image to run actual tempest tests against a live OpenStack environment. Both jobs depend on s2i-openstack-container-content-provider to build the tempest image from source via the buildset registry. Depends-On: openstack-k8s-operators/s2i-openstack-containers#26 Depends-On: openstack-k8s-operators/s2i-openstack-containers#66 Co-authored-by: Cursor <cursoragent@cursor.com>

Add two non-voting jobs that validate the openstack-tempest container image built by s2i-openstack-containers: 1. KUTTL job (test-operator-kuttl-s2i-tempest): runs only the tempest-s2i suite, pins spec.containerImage to the s2i image, and asserts the tempest pod in the kuttl namespace uses that image. 2. E2E job (podified-multinode-edpm-deployment-crc-test-operator-s2i-tempest): full CRC deployment using the s2i-built tempest image to run actual tempest tests against a live OpenStack environment. Both jobs depend on s2i-openstack-container-content-provider to build the tempest image from source via the buildset registry. Depends-On: openstack-k8s-operators/s2i-openstack-containers#26 Depends-On: openstack-k8s-operators/s2i-openstack-containers#66 Co-authored-by: Cursor <cursoragent@cursor.com>

rebtoor mentioned this pull request Aug 13, 2026

Add consumer smoke job and deployment key validation #53

Merged

rebtoor mentioned this pull request Aug 14, 2026

Add e2e job for s2i-built tempest container image openstack-k8s-operators/test-operator#492

Open

4 tasks

rebtoor requested changes Aug 14, 2026

View reviewed changes

openshift-ci Bot assigned rebtoor Aug 14, 2026

amoralej reviewed Aug 14, 2026

View reviewed changes

SeanMooney force-pushed the split/pr14-zuul-provider branch from 028d932 to 1c59201 Compare August 14, 2026 14:39

SeanMooney changed the base branch from split/pr14-zuul-provider-base to main August 14, 2026 14:39

openshift-ci Bot added do-not-merge/contains-merge-commits and removed do-not-merge/contains-merge-commits labels Aug 14, 2026

SeanMooney force-pushed the split/pr14-zuul-provider branch from 1c59201 to 857fbaf Compare August 14, 2026 14:51

SeanMooney added 3 commits August 14, 2026 16:39

SeanMooney force-pushed the split/pr14-zuul-provider branch from 857fbaf to 822f61e Compare August 14, 2026 15:46

SeanMooney mentioned this pull request Aug 14, 2026

Normalize newer generated lockfiles #81

Closed

openshift-ci Bot added the needs-rebase label Aug 14, 2026

openshift-ci Bot added the do-not-merge/hold label Aug 17, 2026

rebtoor mentioned this pull request Aug 17, 2026

Add a Zuul image content provider #85

Merged

4 tasks

rebtoor mentioned this pull request Aug 18, 2026

Add speculative build support: auto-detect and source staging #91

Merged

5 tasks

SeanMooney closed this Aug 19, 2026

Uh oh!

Conversation

SeanMooney commented Aug 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Validation

Dependencies

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amoralej left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amoralej commented Aug 14, 2026

Uh oh!

openshift-ci Bot commented Aug 14, 2026

Uh oh!

openshift-ci Bot commented Aug 14, 2026

Uh oh!

SeanMooney commented Aug 14, 2026

Uh oh!

openshift-ci Bot commented Aug 14, 2026

Uh oh!

rebtoor commented Aug 17, 2026

Uh oh!

SeanMooney commented Aug 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

SeanMooney commented Aug 12, 2026 •

edited

Loading

amoralej left a comment •

edited

Loading