Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions contrib/systemd/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# systemd unit (optional)

Opt-in way to keep `ltm` recording across reboots instead of running
`sudo ltm start` by hand. `ltm start`/`ltm stop` remain the default, portable
path (see [docs/recording.md](../../docs/recording.md)); nothing here is
required to use ltm.

```bash
go build -o bin/ltm ./cmd/ltm
sudo install -m 0755 bin/ltm /usr/bin/ltm
sudo install -m 0644 contrib/systemd/ltm.service /etc/systemd/system/ltm.service
sudo systemctl daemon-reload
sudo systemctl enable --now ltm
```

See [docs/recording.md](../../docs/recording.md#systemd-optional) for
querying, uninstalling, and the unprivileged-user variant.
36 changes: 36 additions & 0 deletions contrib/systemd/ltm.service
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
[Unit]
Description=ltm always-on activity recorder (eBPF)
Documentation=https://github.com/Agent-Hellboy/ltm/blob/main/docs/recording.md
# Recording is Linux/x86_64 only; see docs/recording.md.
After=network.target

[Service]
Type=simple
ExecStart=/usr/bin/ltm daemon --foreground --db /var/lib/ltm/ltm.db --pidfile /run/ltm/ltm.pid

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Put daemon globals before the subcommand

The repo's CLI hard rule says global flags must precede the subcommand, and daemonArgs forwards them as --db ... --pidfile ... daemon --foreground. This unit puts --db/--pidfile after daemon --foreground; those two work only because runDaemon duplicates them as local flags, but the checked-in unit now teaches an ordering that breaks for global-only flags such as --ignore-path if users customize the service in the same position. Please use /usr/bin/ltm --db ... --pidfile ... daemon --foreground here.

Useful? React with 👍 / 👎.

Restart=on-failure
RestartSec=5s

# Owns /var/lib/ltm (db) and /run/ltm (pidfile); created/removed by systemd
# around the service's lifetime.
StateDirectory=ltm
RuntimeDirectory=ltm
Comment on lines +9 to +16

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Remove unused pidfile configuration and correct StateDirectory lifecycle documentation.

The systemd configuration manages a pidfile and RuntimeDirectory that are explicitly noted as unused by the daemon subcommand. Additionally, the comment incorrectly states that systemd removes StateDirectory around the service's lifetime—if this were true, the SQLite database would be deleted on every restart (systemd only removes RuntimeDirectory).

  • contrib/systemd/ltm.service#L9-L16: Remove --pidfile /run/ltm/ltm.pid from ExecStart, remove RuntimeDirectory=ltm, and update the comment to clarify that StateDirectory persists data across restarts.
  • docs/recording.md#L140-L142: Remove the bullet point or update it to only mention StateDirectory and /var/lib/ltm/, since the pidfile and RuntimeDirectory are unnecessary.
📍 Affects 2 files
  • contrib/systemd/ltm.service#L9-L16 (this comment)
  • docs/recording.md#L140-L142
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@contrib/systemd/ltm.service` around lines 9 - 16, Remove the unused pidfile
argument from ExecStart and remove RuntimeDirectory=ltm in
contrib/systemd/ltm.service; update its StateDirectory comment to state that
/var/lib/ltm persists across service restarts. In docs/recording.md, revise the
affected bullet to mention only StateDirectory and /var/lib/ltm/, removing
pidfile and RuntimeDirectory references.


# Recording needs to attach BPF tracepoints and read kernel perf event info:
# root, or CAP_BPF + CAP_PERFMON + CAP_DAC_READ_SEARCH (see docs/security.md).
# Root is the simplest default and what `sudo ltm start` already assumes;
# switch to the capability form below to run unprivileged.
User=root

# --- Unprivileged alternative (comment out User=root above, uncomment these) ---
# CAP_DAC_READ_SEARCH is required too: the collector reads tracepoint ids from
# /sys/kernel/tracing/events/**/id, which are mode 440 root:root, and neither
# CAP_BPF nor CAP_PERFMON grants DAC bypass. Without it every tracepoint open
# fails with "permission denied" and the daemon exits ("no tracepoints could
# be attached"). Verified on a 6.8 kernel.
# User=ltm
# CapabilityBoundingSet=CAP_BPF CAP_PERFMON CAP_DAC_READ_SEARCH
# AmbientCapabilities=CAP_BPF CAP_PERFMON CAP_DAC_READ_SEARCH
# NoNewPrivileges=yes

[Install]
WantedBy=multi-user.target
4 changes: 4 additions & 0 deletions docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,10 @@ sudo ltm stop
the process survives the launching shell. Needs root / BPF caps (see
[security](security.md)).

For always-on recording across reboots, an optional systemd unit
(`daemon --foreground` as `Type=simple`) is available instead — see
[recording.md#systemd-optional](recording.md#systemd-optional).

### `timeline`

Newest-first event list with AND filters. Repeatable flags OR within that field.
Expand Down
56 changes: 56 additions & 0 deletions docs/recording.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,3 +92,59 @@ sudo ltm stop # join producer, close ingest, final flush, then exit
On shutdown the service joins the collector before closing `ingest`, then waits for
the flush loop to finish before the store is closed; otherwise the last batch can be
lost. That path is covered by daemon tests.

## systemd (optional)

`ltm start` is the default, portable way to run the recorder. Machines that
want it always on across reboots can instead run `daemon --foreground`
directly under systemd, using the unit in
[`contrib/systemd/ltm.service`](../contrib/systemd/ltm.service). This is
opt-in and Linux-only (recording already requires Linux); it doesn't replace
`ltm start`/`ltm stop`.

```bash
go build -o bin/ltm ./cmd/ltm
sudo install -m 0755 bin/ltm /usr/bin/ltm
sudo install -m 0644 contrib/systemd/ltm.service /etc/systemd/system/ltm.service
sudo systemctl daemon-reload
sudo systemctl enable --now ltm
```

```bash
systemctl status ltm # unit + process state, journal tail
sudo journalctl -u ltm -f # follow recorder logs
ltm --db /var/lib/ltm/ltm.db status # event counts, dropped, last event
ltm --db /var/lib/ltm/ltm.db timeline --since 10m

sudo systemctl stop ltm # graceful: same shutdown path as `ltm stop`
sudo systemctl disable ltm
```

Notes:

- The unit runs `daemon --foreground` as `Type=simple`, which is what it's
designed for: no forking, no re-exec, systemd tracks the process directly
and restarts it on failure.
- Needs root, or `CAP_BPF` + `CAP_PERFMON` **and** `CAP_DAC_READ_SEARCH` — the

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Add CAP_DAC_READ_SEARCH to Requirements

This new note correctly says non-root recording needs CAP_DAC_READ_SEARCH, but the main Requirements section at the top of this same file still tells users that CAP_BPF + CAP_PERFMON is enough. Someone following the general ltm start docs rather than the later systemd section will configure a recorder that this paragraph says exits after all tracepoint opens fail, so the Requirements bullet should be updated in the same capability correction.

Useful? React with 👍 / 👎.

collector also reads tracepoint ids from
`/sys/kernel/tracing/events/**/id`, which are mode `440 root:root`, so the
two BPF caps alone aren't enough for a non-root user (every tracepoint open
fails with "permission denied" and the daemon exits). The unit defaults to
`User=root` (matching `sudo ltm start`) with a commented-out unprivileged
variant carrying all three capabilities, verified on a 6.8 kernel. See
[security.md](security.md).
Comment on lines +128 to +135

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Sync capability requirements.

The addition of CAP_DAC_READ_SEARCH here is correct, but the ## Requirements section at the top of this file (around line 11) still lists only root, or CAP_BPF + CAP_PERFMON.

Please update the requirements list at the top of the file as well to ensure consistency and prevent users from encountering permission-denied errors when running ltm start directly.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/recording.md` around lines 128 - 135, Update the top-level Requirements
section in recording.md to include CAP_DAC_READ_SEARCH alongside CAP_BPF and
CAP_PERFMON, matching the capability requirement documented in the collector
privilege guidance. Preserve the existing root alternative and formatting.

- The `daemon` subcommand (unlike `start`) doesn't write a pidfile, so
`ltm status`'s `alive` field won't reflect a systemd-managed process — use
`systemctl status ltm` for liveness instead. Event counts/timeline/etc. via
`ltm` still work against the same `--db` path either way.
- `StateDirectory=ltm` / `RuntimeDirectory=ltm` put the db under
`/var/lib/ltm/` and the (unused) pidfile path under `/run/ltm/`; systemd
creates/owns both.
- **Don't point `ltm start` (or a second instance) at the same `--db` as an
active systemd-managed recorder.** Two writers on one SQLite db contend
until a write blows past an internal deadline, both processes exit with
`context deadline exceeded`, and `Restart=on-failure` crash-loops the
service until only one recorder remains. No data corruption results
(`PRAGMA integrity_check` stays `ok`), but it's a real outage of recording
in the meantime — use a distinct `--db` for anything ad hoc, or stop the
unit first.
8 changes: 7 additions & 1 deletion docs/security.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,15 @@ Related: [recording](recording.md) · [querying](querying.md) ·

| Action | Needs |
|---|---|
| `ltm start` (record) | root, or `CAP_BPF` + `CAP_PERFMON` |
| `ltm start` (record) | root, or `CAP_BPF` + `CAP_PERFMON` + `CAP_DAC_READ_SEARCH` |
| timeline / watch / diff / query / sql / status | none — opens the DB read-only |

`CAP_DAC_READ_SEARCH` is needed alongside the two BPF capabilities because the
collector reads tracepoint ids from `/sys/kernel/tracing/events/**/id`, which
are mode `440 root:root`; `CAP_BPF`/`CAP_PERFMON` don't grant DAC bypass on
their own. Without it every tracepoint fails to open and the daemon exits
with "no tracepoints could be attached" (verified on a 6.8 kernel).

`start` re-execs as `daemon --foreground` and detaches (`Setsid`) so the
recorder survives the launching shell.

Expand Down
Loading