EAI-7888 Fix bloom logrotate disabling all log rotation

[pre]

deploy_cluster/logrotate.yaml carries three defects, all present since the
task was written and all reproducible on any bloomed node. Found while
triaging why the app-install-test-cpu-1 post-reboot smoke test failed.

Jira: https://silogen.atlassian.net/browse/EAI-7888

1. The daily logrotate.service fails, so nothing is rotated

Bloom wrote iscsi-aggressive.conf and rke2.conf into /etc/logrotate.d,
where the distro's own rsyslog config already claims /var/log/syslog and
/var/log/kern.log. logrotate refuses a duplicate log entry and aborts the
entire run:

error: rsyslog:1 duplicate log entry for /var/log/syslog
error: rsyslog:1 duplicate log entry for /var/log/kern.log
error: found error in file rsyslog, skipping
logrotate.service: Failed with result 'exit-code'

Nothing under /etc/logrotate.d gets rotated as a result, including the
distro's own rsyslog rotation. Confirmed by A/B test on a live node: the same
config set with bloom's two files removed reports no errors, and putting them
back reproduces the failure exactly.

Both configs are driven by /etc/cron.d/logrotate-bloom with an explicit
path, so moving them to /etc/logrotate-bloom.d/ (outside the include) is
enough. Old installs get the stale files removed.

2. The aggressive iSCSI rotation has never rotated anything

/var/log is root:syslog 775, and a bare logrotate -f <file> never reads
the su root adm in /etc/logrotate.conf. Every cron run since install has
ended in:

error: skipping "/var/log/syslog" because parent directory has insecure permissions

/var/log/logrotate-bloom.log shows this repeating every 10 minutes. Fixed by
putting su root adm in the config itself.

3. The rsyslog iSCSI rate limit was never in effect

01-iscsi-filter.conf failed rsyslog config validation on two counts:

• imuxsock is already loaded by /etc/rsyslog.conf, and the parameter was
  spelled SysSockRateLimit.Interval rather than SysSock.RateLimit.Interval
• the omfile action set both file and dynaFile, so rsyslog looked for a
  template named default, did not find one, and disabled the action

rsyslogd: module 'imuxsock' already in this config, cannot be added
rsyslogd: parameter 'SysSockRateLimit.Interval' not known -- typo in config file?
rsyslogd: omfile: both "file" and "dynafile" set, will use dynafile
rsyslogd: Could not find template 1 'default' - action disabled

Also drops a duplicate enabled: yes key on the cron service task.

4. Older installs keep a second, extensionless copy

The first version of this task installed the configs without a .conf suffix,
documented as cp ... /etc/logrotate.d/iscsi-aggressive. A later version
switched to .conf and left the old files in place. int-test-001 and -002
carry all five:

iscsi-aggressive  iscsi-aggressive.bak  iscsi-aggressive.conf  rke2  rke2.conf

Each duplicates the same log paths, and one stale copy is enough to abort the
whole run. Removing only the .conf pair left this behind:

error: iscsi-aggressive.conf:2 duplicate log entry for /var/log/kern.log
error: rke2.conf:1 duplicate log entry for .../containerd.log

So the removal list covers every name this task has ever written. Found by
running the fix against a second cluster, see below.

Verification

Applied through this task itself, via ansible, to two clusters on two Ubuntu
releases. app-install-test is 22.04 Jammy, int-test is 24.04 Noble, and the
defect is identical on both, which is expected since bloom writes the same
config everywhere.

check	app-install-test (3 nodes)	int-test (3 nodes)
rsyslogd -N1	0 errors	0 errors
systemctl is-active rsyslog	active	active
logrotate --debug /etc/logrotate.conf	0 errors	0 errors
systemctl start logrotate.service	succeeds	succeeds
stale files left in /etc/logrotate.d	none	none
both cron configs run	rc=0	rc=0

Before the fix, all six nodes had logrotate.service in failed and
rsyslogd -N1 reporting 3 errors.

go build ./... and go test ./pkg/... pass.

[pre]

Verified on two fresh Kaytoo VMs (A = main baseline, B = this branch)

Both bloom binaries were built from source, since the playbooks are embedded in the binary and a release tag would not carry the fix. The only difference between the two runs is the branch commit. CLUSTERFORGE_RELEASE: none, since the logrotate/rsyslog tasks run in bloom's own deploy_cluster stage.

	VM A (baseline)	VM B (fix)
bloom built from	main @ 6511f52	fix-logrotate-rsyslog @ 00b47d8
PLAY RECAP	ok=195 changed=83 failed=0	ok=197 changed=84 failed=0

B's two extra tasks are the new logrotate-bloom.d directory and the old-config cleanup.

1. 01-iscsi-filter.conf rsyslog parse errors

VM A, three distinct errors, present both in rsyslogd -N1 and in the live journal at service start:

module 'imuxsock' already in this config, cannot be added
error ... 01-iscsi-filter.conf, on or before line 2: parameter 'SysSockRateLimit.Interval' not known -- typo in config file?
error ... 01-iscsi-filter.conf, on or before line 13: omfile: both "file" and "dynafile" set, will use dynafile
error ... errors occurred in file '/etc/rsyslog.d/01-iscsi-filter.conf' around line 13

VM B: rsyslogd -N1 clean, journal clean, rsyslog starts with no complaint.

2. logrotate aborting the whole daily run

On A the bloom configs sit in /etc/logrotate.d/, so the distro's own rsyslog config collides with them:

error: rsyslog:1 duplicate log entry for /var/log/syslog
error: rsyslog:1 duplicate log entry for /var/log/kern.log
error: found error in file rsyslog, skipping

Starting the real daily unit on A gives Result=exit-code, ExecMainStatus=1. On B the configs live in /etc/logrotate-bloom.d/, outside the include, the run is clean, and the unit gives Result=success, ExecMainStatus=0.

3. The bloom cron job actually rotating

On A the forced run, the exact command in /etc/cron.d/logrotate-bloom, exits 1 and rotates nothing:

error: skipping "/var/log/syslog" because parent directory has insecure permissions ...

syslog and kern.log are untouched. On B the same run exits 0 and the rotation happens: syslog-20260806 and kern.log-20260806 created, live files back to 0 bytes. That is the su root adm line.

4. Migration path

B has no iscsi-aggressive.conf or rke2.conf left in /etc/logrotate.d/, so the cleanup task removes what earlier bloom versions wrote. Verified on a fresh node only, not on a node upgraded from an older bloom, though the task is an unconditional state: absent so it does not depend on the prior state.

Executive summary: the test did work, and the fix is confirmed.

Both VMs installed cleanly (failed=0), and the two-way comparison held:

• VM A (main) reproduced the bug — the 01-iscsi-filter.conf rsyslog parse errors were there, the daily logrotate.service failed with exit 1, and bloom's own cron job rotated nothing.
• VM B (branch) had none of it — rsyslog config clean, logrotate.service exits 0, and syslog/kern.log actually rotated.

The only difference between the two runs was the branch commit, so the change is what caused the difference.