Use unix sockets for httpd -> Foreman and Pulp communication

[Gauravtalreja1]

Why are you introducing these changes? (Problem description, related links)

On hosts without IPv4 (IPv6-only), Apache resolves localhost to ::1 but Puma/Gunicorn only listen on 127.0.0.1, causing connection refused errors:

AH00957: http: attempt to connect to [::1]:3000 (localhost:3000) failed

This affects all three httpd backends: Foreman (port 3000), Pulp API (port 24817), and Pulp Content (port 24816).
Builds on the work started by @ekohl in #118 and #593. In foreman-installer, using a unix socket had reliability benefits, this brings the same approach to foremanctl.

Why unix sockets instead of binding to 127.0.0.1 explicitly:

• Parity with foreman-installer: puppet-foreman and puppet-pulpcore already use unix sockets
• No TCP port conflicts: doesn't occupy TCP ports 3000, 24816, 24817
• Filesystem-level access control: socket permissions via SocketUser/SocketMode
• Eliminates TCP loopback overhead
• Systemd socket activation: clean dependency ordering via Requires=

What are the changes introduced in this pull request?

Foreman (cherry-picked from #593, authored by @ekohl):

• Systemd socket activation for Foreman
• Unix socket between httpd and Puma at /run/httpd.foreman.sock

SELinux fixes (on top of #593):

• Add missing when guard for daemons_enable_cluster_mode boolean
• Remove httpd_can_network_connect, no longer needed with all backends on unix sockets

Pulp:

• Unix sockets for Pulp API (/run/httpd.pulp-api.sock) and Pulp Content (/run/httpd.pulp-content.sock)
• Systemd socket units for both services
• Container command overrides to bind Gunicorn to unix sockets

How to test this pull request

• Deploy using foremanctl on an IPv6-only or IPv4-only host
• Verify no TCP listeners on ports 3000, 24816, 24817: ss -tlnp | grep -E '(3000|24816|24817)'
• Verify unix sockets exist: ls -laZ /run/httpd.*.sock
• The Foreman UI loads
• Pulp API responds: curl -k https://$(hostname -f)/pulp/api/v3/status/
• Test suite passes

Steps to reproduce:

Checklist

• Tests added/updated (if applicable)
• Documentation updated (if applicable)

[ekohl]

theforeman/puppet-pulpcore@04b9f5f may be an interesting reference.

[evgeni]

I've rebased this now that #634 is merged, to be able to debug what's happening in the external proxy test case.

[ekohl]

Looks like Apache sends the literal %{http_host} header, not interpolated. Now I wonder if that's also a problem in our regular install.

ul 08 07:41:50 proxy.example.com pulp-api[45255]: pulp [9d33dfa3a8194a68ad9b224b1fc1de0c]: django.security.DisallowedHost:ERROR: Invalid HTTP_HOST header: '%{http_host}'. The domain name provided is not valid according to RFC 1034/1035.
Jul 08 07:41:50 proxy.example.com pulp-api[45255]: Traceback (most recent call last):
Jul 08 07:41:50 proxy.example.com pulp-api[45255]:   File "/usr/lib/python3.12/site-packages/django/core/handlers/exception.py", line 55, in inner
Jul 08 07:41:50 proxy.example.com pulp-api[45255]:     response = get_response(request)
Jul 08 07:41:50 proxy.example.com pulp-api[45255]:                ^^^^^^^^^^^^^^^^^^^^^
Jul 08 07:41:50 proxy.example.com pulp-api[45255]:   File "/usr/lib/python3.12/site-packages/django/utils/deprecation.py", line 133, in __call__
Jul 08 07:41:50 proxy.example.com pulp-api[45255]:     response = self.process_request(request)
Jul 08 07:41:50 proxy.example.com pulp-api[45255]:                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jul 08 07:41:50 proxy.example.com pulp-api[45255]:   File "/usr/lib/python3.12/site-packages/django/middleware/common.py", line 48, in process_request
Jul 08 07:41:50 proxy.example.com pulp-api[45255]:     host = request.get_host()
Jul 08 07:41:50 proxy.example.com pulp-api[45255]:            ^^^^^^^^^^^^^^^^^^
Jul 08 07:41:50 proxy.example.com pulp-api[45255]:   File "/usr/lib/python3.12/site-packages/django/http/request.py", line 151, in get_host
Jul 08 07:41:50 proxy.example.com pulp-api[45255]:     raise DisallowedHost(msg)
Jul 08 07:41:50 proxy.example.com pulp-api[45255]: django.core.exceptions.DisallowedHost: Invalid HTTP_HOST header: '%{http_host}'. The domain name provided is not valid according to RFC 1034/1035.

[ekohl]

@Gauravtalreja1 #608 was merged. Can you please rebase it to resolve the merge conflicts?

[evgeni]

#471 was merged, so if we rebase here and fix the hostnames, we should see if things work properly

[ehelms]

On hosts without IPv4 (IPv6-only), Apache resolves localhost to ::1 but Puma/Gunicorn only listen on 127.0.0.1, causing connection refused errors:

AH00957: http: attempt to connect to [::1]:3000 (localhost:3000) failed

As I understand it, there are other ways to solve this issue. Can you expand on why we want to go with the unix socket solution? Are there other things that this choice solves or provides? I am not opposed to the change, but I would find it helpful to capture more about this choice in the pull request description and in our documentation given this is an explicit design choice we want to have / enforce.

[coderabbitai]

Walkthrough

This change introduces systemd socket units for Foreman, Pulp API, and Pulp Content services, deployed via new Ansible templates and tasks, and adds explicit Requires dependencies in their container quadlet units. New listen-stream variables define UNIX socket paths, and httpd backend variables are derived from them in base vars. The httpd role's SELinux boolean configuration is updated to replace network-connect permission with cluster-mode and conditional network-relay permissions. Tests for Foreman and Pulp are updated to verify UNIX socket connectivity instead of TCP port reachability.

Estimated code review effort: 3 (Moderate) | ~25 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: switching httpd communication for Foreman and Pulp to Unix sockets.
Description check	✅ Passed	The description is directly related to the changeset and explains the Unix socket migration and SELinux adjustments.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[Gauravtalreja1]

On hosts without IPv4 (IPv6-only), Apache resolves localhost to ::1 but Puma/Gunicorn only listen on 127.0.0.1, causing connection refused errors:

AH00957: http: attempt to connect to [::1]:3000 (localhost:3000) failed

As I understand it, there are other ways to solve this issue. Can you expand on why we want to go with the unix socket solution? Are there other things that this choice solves or provides? I am not opposed to the change, but I would find it helpful to capture more about this choice in the pull request description and in our documentation given this is an explicit design choice we want to have / enforce.

@ehelms Good point, I've updated the PR description with the design rationale (with what I understand) for choosing unix sockets over binding to 127.0.0.1/localhost.
For broader context on the original design decision, @ekohl drove this approach in #118 & #593 and can provide more insight into the long-term vision behind it.

[evgeni]

test_https_rhsm_proxy is failing, which is the only test that uses the "httpd-on-proxy is a reverse proxy to httpd-on-server which in turn is a reverse proxy to rails" chain.

I bet this is some cross play with #471 - but that test was green there, so it's something about the sockets 🙈

logs on the proxy say:

[Thu Jul 09 15:14:00.089578 2026] [proxy:error] [pid 44340:tid 44420] (13)Permission denied: AH00957: https: attempt to connect to 192.168.124.97:443 (quadlet.example.com:443) failed
[Thu Jul 09 15:14:00.089633 2026] [proxy_http:error] [pid 44340:tid 44420] [client 192.168.124.97:59898] AH01114: HTTP: failed to make connection to backend: quadlet.example.com

192.168.124.97 - - [09/Jul/2026:15:14:00 +0000] "GET /rhsm HTTP/1.1" 503 299 "-" "curl/7.76.1"

I see no requests reaching the quadlet box.

I think you need to put back that httpd network selinux boolean

[Gauravtalreja1]

@coderabbitai full review

[coderabbitai]

✅ Action performed

Full review finished.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/roles/pulp/tasks/main.yaml`:
- Around line 95-100: Both socket template tasks are missing restart
notifications, so changes to the unit files won’t restart already-running
sockets. Update the `Deploy Pulp API socket` and `Deploy Pulp Content socket`
tasks in `main.yaml` to notify dedicated handlers that restart `pulp-api.socket`
and `pulp-content.socket`, and place those handlers so they run before the
container restart handlers. This ensures template changes are applied
idempotently and the containers get a fresh socket file descriptor after any
socket unit update.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 9c7415f2-e9c1-4b13-9b31-c7de5b013c88

📥 Commits

Reviewing files that changed from the base of the PR and between 99670e3 and 941fbe1.

📒 Files selected for processing (11)

• src/roles/foreman/defaults/main.yaml
• src/roles/foreman/tasks/main.yaml
• src/roles/foreman/templates/foreman.socket.j2
• src/roles/httpd/tasks/main.yml
• src/roles/pulp/defaults/main.yaml
• src/roles/pulp/tasks/main.yaml
• src/roles/pulp/templates/pulp-api.socket.j2
• src/roles/pulp/templates/pulp-content.socket.j2
• src/vars/base.yaml
• tests/foreman_test.py
• tests/pulp_test.py

[Satellite-QE]

PRT Result

Build Number: 16084
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/installer/test_install_foremanctl.py --external-logging
Test Result: ================= 34 warnings, 20 errors in 4107.43s (1:08:27) =================

[Satellite-QE]

PRT Result

Build Number: 16087
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/installer/test_install_foremanctl.py --external-logging
Test Result: ================= 34 warnings, 20 errors in 3812.58s (1:03:32) =================

[evgeni]

PRT fails with Unable to start service postgresql: A dependency job for postgresql.service failed. which makes no sense :/

Edit: oooh, it used a v6 machine, but that probably can't reach quay.io…

[Satellite-QE]

PRT Result

Build Number: 16089
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/installer/test_install_foremanctl.py --external-logging
Test Result: ================= 34 warnings, 20 errors in 3905.19s (1:05:05) =================

[ekohl]

Thanks for taking this across the finish line.

[stejskalleos]

Thanks everyone!