Skip to content

Rootless claude - #23

Open
pablomh wants to merge 50 commits into
masterfrom
rootless_claude
Open

Rootless claude#23
pablomh wants to merge 50 commits into
masterfrom
rootless_claude

Conversation

@pablomh

@pablomh pablomh commented Apr 16, 2026

Copy link
Copy Markdown
Owner

Why are you introducing these changes? (Problem description, related links)

What are the changes introduced in this pull request?

How to test this pull request

Steps to reproduce:

Checklist

  • Tests added/updated (if applicable)
  • Documentation updated (if applicable)

@pablomh
pablomh force-pushed the rootless_claude branch 25 times, most recently from 90f6e68 to a408f28 Compare April 17, 2026 08:24
pablomh and others added 4 commits April 17, 2026 10:29
@pablomh @claude
Add foremanctl service subcommand for operational management
fc7b03d 
Introduces `foremanctl service` as a unified operational interface for
managing Foreman services and containers. Delegates to systemctl,
journalctl, and podman directly.

Subcommands: status [-b], start, stop, restart, logs, ps, exec

The -b/--brief flag prints a compact one-liner per service in systemd
startup order (derived from list-dependencies), matching the UX of
foreman-maintain service status -b.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@pablomh @claude
Add test helpers and tests for foremanctl service subcommand
92a69e8 
Adds helpers to conftest.py that delegate to `foremanctl service`,
making all test assertions go through the operational tool:

- service_is_running(server, name) — foremanctl service status
- service_is_enabled(server, name) — systemctl --user is-enabled
- service_exists(server, name)     — systemctl --user status (rc != 4)
- service_start/stop/restart(server, name)
- foremanctl_run(server, cmd)      — run as foremanctl user
- foremanctl_exec(server, ctr, cmd)— foremanctl service exec

Also adds foremanctl_service_test.py which tests the tool itself:
help output, status, brief mode, startup order, ps, unknown subcommand,
and exec.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@pablomh @claude
Adapt existing tests to use foremanctl service API
a600313 
Replaces server.service() and raw podman/systemctl calls with the new
test helpers that delegate to foremanctl service. Tests now exercise
the operational tool at the same time as verifying service state.

- service_is_running() replaces server.service().is_running
- service_is_enabled()/service_exists() for enable/existence checks
- service_start/stop/restart() in target_lifecycle_test
- foremanctl_exec() replaces raw podman exec invocations
- Port reachability tests kept where ports are published to loopback

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@pablomh @claude
Add rootless_user role for rootless Podman deployment
bc7901e 
Creates the foremanctl system user that owns all containerized services
in rootless mode:

- Creates foremanctl group and user (home /var/lib/foremanctl, no login)
- Derives XDG_RUNTIME_DIR from the user's actual UID
- Writes subuid/subgid entries (100000:65536) for user namespace mapping
- Sets net.ipv4.ip_unprivileged_port_start=80 via sysctl
- Enables systemd lingering so services survive without an active session
- Verifies XDG_RUNTIME_DIR exists (fails fast if lingering didn't work
- Creates ~/.config/containers/systemd and ~/.config/systemd/user dirs

Exposes rootless_user_xdg_runtime_dir fact for the deploy playbook to
map into the foremanctl namespace.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
EOF
)
@pablomh @claude
Make postgresql role self-contained for rootful deployments
d7e7267 
The role was converted to rootless-only, breaking development and external
DB contexts that do not set up the foremanctl user or bridge networks.

Add rootful fallback defaults so the role works standalone:
- postgresql_networks defaults to 'host' (original rootful behavior)
- postgresql_systemd_scope defaults to 'system' (rootful systemd scope)
- foremanctl_* vars default to root-appropriate values so become_user
  and runtime_env work when base.yaml is not loaded

The rootless deployment (deploy.yaml) now explicitly opts into rootless
behavior via role call vars (foreman-db network, user scope).

Also revert foremanctl-service run_podman to systemd-run --machine --user
--wait --pipe: propagates exit codes correctly (unlike machinectl shell
which always exits 0), requires no manual XDG_RUNTIME_DIR, and is the
canonical systemd-native mechanism for cross-user podman invocations.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
pablomh and others added 3 commits April 17, 2026 17:19
@pablomh @claude
Fix CI regressions: ansible-lint, podman unshare, foreman-proxy featu…
b5c078a 
…res, installer key

Four fixes:

1. postgresql role: remove foremanctl_* vars from defaults (violated
   var-naming[no-role-prefix] ansible-lint rule). Rootful fallbacks now
   expressed as | default() at every usage site in tasks and handlers.

2. postgresql role: gate podman unshare chown on postgresql_systemd_scope
   == user. podman-unshare is rootless-only; in rootful mode (system scope)
   it fails with 'please use unshare with rootless'.

3. foreman_proxy handler: add daemon_reload: true to Restart Foreman Proxy.
   Without it, new feature drop-in configs in foreman-proxy.container.d/
   are ignored and the proxy starts with zero features at registration.

4. deploy.yaml: change installer private key chmod from 0640 to 0644.
   foremanctl is not in root group so 0640 leaves keys unreadable.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@pablomh @claude
Fix foreman-proxy restart, postgresql rootful ownership, deploy-dev s…
557f33a 
…etup

Three fixes:

1. foreman_proxy handler: replace ternary state logic with unconditional
   'state: restarted'. On first deployment the ternary picked 'started'
   (no-op) because _foreman_proxy_service.changed was true, so the proxy
   never restarted to pick up feature drop-in configs added after the
   initial start. The handler is named Restart — it should always restart.

2. postgresql data dir: default owner/group to postgresql_container_uid/gid
   (26:26) instead of root. In rootful mode the container runs with real
   UIDs so the postgres process (uid 26) needs direct ownership of the
   0700 data directory. In rootless mode foremanctl_user is always defined
   so the default never fires.

3. deploy-dev.yaml: add import_playbook for setup-rootless-user as Play 1,
   same pattern as deploy.yaml. Without it, base.yaml sets foremanctl_user
   to foremanctl but the user does not exist, and the certificates role
   fails with 'chown failed: failed to look up user foremanctl'.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@pablomh @claude
Fix missing | default() in postgresql SSL cert block
581aef5 
The SSL cert block tasks (Create postgresql.container.d folder, Configure
PostgreSQL SSL-related secrets) use foremanctl_user/foremanctl_group for
file ownership but were missed by the earlier | default() sweep because
they use 8-space indentation (inside a block) vs the 4-space pattern
that replace_all matched. In rootful mode (external DB with SSL) these
vars are undefined, causing 'foremanctl_group is undefined'.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@pablomh
pablomh force-pushed the rootless_claude branch 2 times, most recently from 4f69a60 to e197d79 Compare April 17, 2026 22:05
@pablomh @claude
Remove dual-mode postgresql role — always rootless, callers provide c…
f1ebf2c 
…ontext

The postgresql role was accumulating | default() band-aids to support both
rootful and rootless modes. This made the code noisy, fragile (easy to miss
spots), and inconsistent with every other role in the codebase.

Clean approach: the role always runs rootless. Every caller provides the
foremanctl context, same as with redis, candlepin, pulp, foreman roles.

Changes:
- base.yaml: guard certificates_hostnames with 'enabled_features is defined'
  so playbooks that load base.yaml without defining enabled_features (like
  remote-database.yaml) don't fail. This was the root cause of the original
  external DB cert error.
- remote-database.yaml: add rootless user setup (Play 1) and load base.yaml
  (Play 2), matching the proven pattern from commit 5c8ee93 in the
  rootless_iop branch. The foremanctl user now exists on the database host.
- postgresql defaults: restore foreman-db network and user scope as defaults.
- postgresql tasks/handlers: remove all | default() — foremanctl_* vars are
  always defined by the caller via base.yaml.
- deploy.yaml: remove explicit postgresql_networks and postgresql_systemd_scope
  overrides — the role defaults are now correct for the main deployment.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
pablomh and others added 7 commits April 18, 2026 00:12
@pablomh @claude
Add missing pulp podman unshare chown and deploy-dev host networking
ae063c0 
Two fixes found by reviewing against rootless_iop branch:

1. pulp role: add podman unshare chown for container UID 700. Without it,
   the pulp process cannot write to /var/lib/pulp (owned by foremanctl on
   the host, which maps to root inside the container, not uid 700). Redis
   and postgresql already have this; pulp was missed during conversion.

2. deploy-dev.yaml: add host networking overrides for postgresql, redis,
   candlepin, and pulp. The dev environment has no bridge networks (no
   deploy_network roles), so all services must use host networking with
   localhost for database/redis connections.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@pablomh @claude
Fix pulp key generation ordering, external DB host networking
ddf4d71 
Three fixes from reviewing against rootless_iop:

1. pulp role: move Django secret key and database symmetric key generation
   to root-level tasks BEFORE podman unshare chown. After the chown remaps
   /var/lib/pulp to container uid 700, the foremanctl user can no longer
   write files there. Use ansible.builtin.copy with force: false (matching
   rootless_iop pattern) instead of shell redirect.

2. pulp role: add pulp_container_uid/gid defaults (700:700) for the podman
   unshare chown task added in the previous commit.

3. remote-database.yaml: add postgresql_networks: host to the postgresql
   role call. The database host has no bridge networks — foreman-db does
   not exist there. Without this override the container fails to start.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@pablomh @claude
Replace localhost with container DNS names in service templates
b884b54 
In bridge networking, localhost inside a container refers to the container
itself, not the host or other containers. Inter-container connections must
use Podman DNS names.

- katello.yaml.j2: candlepin URL and broker_host localhost -> candlepin
- settings.yaml.j2: Redis cache URL localhost -> redis
- broker.xml.j2: Artemis STOMP acceptor localhost -> {{ candlepin_hostname }}

All three match the rootless_iop branch patterns.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@pablomh @claude
Fix migration container networking for external DB mode
8d736bb 
Pulp and Foreman migration containers (db:migrate, admin-password) hardcode
foreman-db as their network. foreman-db is internal/isolated — no external
routing. In external DB mode, PostgreSQL is on a remote host that can only
be reached via foreman-app (which has external routing).

Use inline conditional matching rootless_iop:
  network: ['foreman-db'] if database_mode == 'internal' else ['foreman-app']

Applied to: pulpcore-manager-migrate, pulpcore-manager-admin-password,
foreman-db-migrate.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@pablomh @claude
Add pulp_migration_networks variable for context-dependent DB routing
d1b8c2a 
The inline conditional for migration container networking didn't handle
the dev environment where no bridge networks exist. Extract to a role
default variable (pulp_migration_networks) matching rootless_iop's pattern:
- Internal DB: foreman-db (isolated bridge)
- External DB: foreman-app (external routing)
- Dev: host (overridden in deploy-dev.yaml, no bridge networks)

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@pablomh @claude
Add daemon_reload to postgresql restart handler for SSL drop-in
542534d 
The SSL drop-in config (postgresql.container.d/ssl.conf) was added to
master after rootless_iop branched, so the reference branch never had
this issue. Without daemon_reload, the SSL secret mounts from the drop-in
are not picked up on restart, causing 'server does not support SSL' in
the external DB test.

Same fix pattern as the foreman-proxy handler.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@pablomh @claude
Replace foremanctl CLI calls with systemctl --machine in test helpers
97b2d29 
The foremanctl command is not installed inside the VM — it runs on the
CI runner as an obsah wrapper. Test helpers that called 'foremanctl
service ...' failed with 'command not found'.

Replace with the underlying systemd-native commands that foremanctl-service
wraps:
- systemctl --machine=foremanctl@ --user for service operations
- systemd-run --machine=foremanctl@ --user --wait --pipe -- podman exec
  for container exec

This also eliminates the D-Bus dependency: --machine= uses systemd-machined
protocol, not the session bus, so it works regardless of how the test
framework switches to the foremanctl user context.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
pablomh and others added 3 commits April 18, 2026 16:36
@pablomh @claude
Use PostgreSQL 16 and Valkey 8 from CentOS 10 Stream
ef725eb 
Apply upstream PR theforeman#395: switch from c9s containers (postgresql-13, redis-6)
to c10s containers (postgresql-16, valkey-8) matching the RHEL 10 target.

- postgresql-13-c9s -> postgresql-16-c10s
- redis-6-c9s -> valkey-8-c10s
- run-redis -> run-valkey (new entrypoint command)

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@pablomh @claude
Fix candlepin and certificate tests for bridge networking
632f178 
- test_candlepin_status: use curl --resolve candlepin:23443:127.0.0.1 so
  TLS verification matches the candlepin cert SAN (not localhost)
- test_artemis_port: replace ss (not in container image) with bash
  /dev/tcp built-in for port check
- test_certificate_expiry: replace localhost_certificate (no longer
  generated) with candlepin_certificate; add skip for unconfigured certs

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@pablomh @claude
Replace ALTER SYSTEM with sclorg config mount for socket directories
5615851 
The ALTER SYSTEM + restart cycle for unix_socket_directories fails with
the PG16 c10s container image. The second restart (after ALTER SYSTEM
writes to postgresql.auto.conf) causes the container to exit with an
error code.

Replace with a Podman secret mounted at /opt/app-root/src/postgresql-cfg/
socket.conf — the sclorg-native mechanism for custom PostgreSQL settings.
This is the same approach already used for SSL configuration. The socket
config is applied at first start, eliminating the restart cycle entirely.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@pablomh
pablomh force-pushed the rootless_claude branch 5 times, most recently from ed155c0 to dbda8e7 Compare April 18, 2026 22:19
@pablomh @claude
Mount socket dir at /tmp instead of /tmp/socket
c56d756 
The sclorg default unix_socket_directories already includes /tmp, so
mounting the host socket dir at /tmp inside the container provides the
Unix socket without any custom configuration — no ALTER SYSTEM, no
config secret, no extra restart cycle.

The ALTER SYSTEM approach failed with PG16 c10s because the setting in
postgresql.auto.conf replaced the sclorg default (which includes /tmp
for backwards compatibility), causing the healthcheck or entrypoint to
fail on restart.

The sclorg entrypoint chown issue (documented in MEMORY.md) only affects
/var/run/postgresql, not /tmp, so this mount path is safe.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@pablomh @claude
Fix dynflow symlink test: use absolute path for systemd-run
47599f5 
systemd-run --machine= runs commands directly without a shell, so ~
does not expand to the foremanctl home directory. Use the absolute path
/var/lib/foremanctl/.config/containers/systemd/ instead.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@pablomh