New api endpoint notification history

[Jan-Schuppik]

Sources currently have no way to ask Icinga Notifications which notifications it actually attempted for their objects. This PR adds a GET /notification-history HTTP endpoint that streams every notification attempt (sent or failed) recorded since a given point in time, along with the database schema and internal plumbing needed to record that history.

What's included

Two new tables, notification_history and skipped_notification_history, record every notification attempt and, for cases where multiple escalation recipients would have notified the same contact through the same channel, the additional origins that were skipped as duplicates. To support this, rule.ContactChannels was reworked from a per-contact/channel boolean into a map of []ChannelOrigin (rule, escalation, contact group, schedule, role), so internal/incident can tell which origin a pending notification is attributed to and which ones were superfluous.

Events now carry a stable event_id (a UUID either provided by the source or derived by hashing the event's JSON representation via the new event.CreateEvent/EnsureID). This ID is persisted on incident_history and used to correlate notification_history rows to the event that triggered them. As part of this, the event_queue deduplication key was migrated from a raw SHA256 bytea hash to the same UUID.

ContactRole and NotificationState moved out of internal/incident: ContactRole now lives in internal/recipient (its file relocated to internal/utils/contact_role.go), and the local NotificationState type was replaced by icinga-go-library's notifications/source.NotificationState, which required bumping the icinga-go-library dependency in go.mod.

The new endpoint reuses the same source-authentication and NDJSON-streaming machinery as the existing /incidents endpoint; the shared error-handling logic in listener.go was factored out into createStreamErrFunc so both endpoints use it.

Schema changes (see schema/{mysql,pgsql}/upgrades/notification-history.sql): adds the notification_history and skipped_notification_history tables, adds incident_history.event_id, and changes event_queue.id from a binary hash column to a UUID column. A running deployment needs the migration applied before upgrading.

Testing

Added unit tests for event ID generation (event_id_test.go), the event queue (queue_test.go), notification history persistence (notification_history_test.go, requires a test database), and the new listener endpoint (listener/notification_history_test.go).

To verify manually, run the daemon, trigger some events, then query the endpoint as documented in the new doc/20-HTTP-API.md section:

curl -u 'example:insecureinsecure' 'http://localhost:5680/notification-history?since=<unix-ms>'

Depends on:

• notifications: support /notification-history API endpoint icinga-go-library#229

resloves #415

[yhabteab]

I haven't got the time to review this thoroughly, but I left some inline comments below. I'll try to get back to this tomorrow and do a more thorough review.

[yhabteab]

Another round of review comments :). Sorry for the chunked review comments but the PR is quite big, so I need to do something else to get my head right again :). Apart from the tests, I've now reviewed all at least once.

[yhabteab]

Apart from the below comment, I think I'm done reviewing the non-test code now.

[yhabteab]

Ah, sorry. I also forget to mention that you've to adjust the retention pruner for the object table to consider the notification_history reference.