ADR-0001: Re-enable LLM-Dev-Ops ecosystem dependencies

[nicholas-ruest]

Implements docs/adr/ADR-0001-re-enabling-llm-dev-ops-ecosystem-dependencies.md (added in this PR — it was untracked).

TL;DR — the ADR cannot be executed yet, and this PR says so precisely

ADR-0001 decided to re-enable the five ecosystem dependencies as tag-pinned git dependencies behind a working ecosystem feature. Its Implementation Plan step 1 (upstream tags) is a blocking precondition, and the ADR is explicit: "incident-manager does not merge step 1 until each upstream repo has cut a tag whose cargo build succeeds standalone on Rust 1.84."

I checked all five against the real remotes. Step 1 is unmet for all five. So this PR re-enables nothing, and instead replaces the misleading manifest comment with per-dependency status and evidence.

Per-dependency status (verified 2026-07-27)

Dependency	Repo	Status
llm-observatory-core	observatory	0 tags, 0 releases — nothing immutable to pin
llm-governance-common	governance-dashboard	0 tags, 0 releases — nothing immutable to pin
llm-analytics-hub	analytics-hub	0 tags, 0 releases — nothing immutable to pin
policy-engine-benchmarks	policy-engine	0 tags, 0 releases — nothing immutable to pin
llm-sentinel-core	sentinel	has tag v0.1.0, but it does not build

Checked with gh api repos/LLM-Dev-Ops/<repo>/tags and git ls-remote --tags.

sentinel v0.1.0 — pinned and compiled, not assumed

I actually added the pin and ran the build rather than taking the tag's existence as sufficient. Two independent defects:

1. sentinel-core does not compile against validator 0.18.

   error[E0599]: no method named `validate_length` found for struct `ServiceId` in the current scope
      --> crates/sentinel-core/src/events.rs:16:48
   16 | #[derive(Debug, Clone, Serialize, Deserialize, Validate)]
      |                                                ^^^^^^^^
   142 | pub struct ServiceId(String);
       | -------------------- method `validate_length` not found for this struct
   error: could not compile `sentinel-core` (lib) due to 1 previous error
   

   events.rs applies #[validate(length(min = 1, max = 255))] to service_name: ServiceId, but ServiceId/ModelId are newtypes that don't implement validator 0.18's ValidateLength.

2. The workspace doesn't even resolve standalone, so it fails ADR step 1's acceptance criterion outright:

   error: failed to select a version for the requirement `axum-test = "^16.5"`
   candidate versions found which didn't match: 21.0.0, 20.1.0, 20.0.0, ...
   required by package `sentinel-api v0.1.0`
   

   16.x is gone from crates.io, and there is no Cargo.lock at the tag to hold it.

Bonus trap: at v0.1.0 the crate is named sentinel-core, not llm-sentinel-core — the rename happened after the tag. ADR-0001's target-state line llm-sentinel-core = { git = ..., tag = "vX.Y.Z" } would fail to resolve as written; it needs package = "sentinel-core" unless a new tag is cut post-rename.

ADR-0001 step 2 forbids papering over any of this with [patch], so it must be fixed upstream.

What this PR actually changes

• Cargo.toml — replaces "temporarily disabled due to upstream dependency issues" with per-dependency status, the concrete sentinel failures, and the rule that any re-enabled line must carry an immutable tag.
• [features] — deletes the false claim that "When disabled (default), stub implementations are used." There is no such switch; grep -rn 'feature = "ecosystem"' src/ returns nothing (ADR Finding 2). ecosystem = [] is documented as a known, deliberate no-op (Finding 1) that cannot name a dependency until one is pinnable.
• Three adapters — sentinel_adapter.rs, observatory_adapter.rs, analytics_adapter.rs blocker comments now name the specific missing tag / defect instead of "Uncomment when ecosystem dependencies are available".
• The ADR itself is committed (it was untracked).

Deliberately no functional change: the default build is byte-for-byte the same dependency graph as before.

Build / test output (real)

cargo build --workspace                     -> Finished dev profile in 2m 45s   EXIT=0
cargo build --workspace --features ecosystem -> Finished dev profile in 4m 17s   EXIT=0

cargo tree | grep -c 'sentinel-core\|llm-observatory\|llm-governance\|llm-analytics\|policy-engine'
  -> 0        (ADR check 3: default build unchanged)

cargo test --lib integrations::upstream
  -> test result: ok. 22 passed; 0 failed; 0 ignored; 456 filtered out
cargo test --lib --features ecosystem integrations::upstream
  -> test result: ok. 22 passed; 0 failed; 0 ignored; 456 filtered out

Caveat, stated plainly: the full cargo test --workspace does not complete in my environment. It compiles, starts running 478 tests, then hangs indefinitely on a test holding open sockets (~30 min, 0% CPU, blocked on futex) — evidently a test expecting Redis/NATS/Kafka to be reachable. This is pre-existing and unrelated to this PR (the diff is comments only), but I could not run the full suite to completion and am not claiming otherwise. I scoped the run to integrations::upstream, the modules this PR touches.

Also note: cargo build --workspace fails on a clean checkout of main in a bare container until you install cmake, protobuf-compiler, libcurl4-openssl-dev, clang, libsasl2-dev, zlib1g-dev, libssl-dev — rdkafka builds librdkafka from source. Worth knowing for CI.

ADR-0001 verification checklist — honest scorecard

#	Check	Result
1	grep -c '^# llm-' Cargo.toml == 0	PASS (0)
2	every git = carries tag =, no branch =	PASS vacuously — no git deps exist
3	default build succeeds, 0 LLM-Dev-Ops crates in tree	PASS
4	--features ecosystem pulls the four runtime crates	FAIL — blocked upstream
5	no duplicate llm-* versions	PASS vacuously
6	cfg(feature = "ecosystem") + cfg(not(...)) in all three adapters	FAIL — blocked upstream
7	both test configurations pass, incl. test_outlier_type_conversion	PARTIAL — adapter tests pass in both; that test still needs analytics-hub
8	--features benchmarks builds	FAIL — feature can't exist without a policy-engine tag
9	CI runs all four configurations	NOT DONE — see below
10	upstream main moving doesn't change our graph	PASS vacuously

I deliberately avoided writing #[cfg(feature = "ecosystem")] verbatim inside the new comments, because check 6 is a grep and prose in a comment would have satisfied it spuriously.

Things needing a human decision

1. Four upstream repos must cut tags; sentinel must fix two defects and cut a new one. incident-manager cannot do this — the ADR assigns it to upstream owners. I did not create tags on other repos.
2. Cargo.lock is gitignored (.gitignore:5). Every ADR-0001 verification step specifies cargo build --locked, which cannot work here. A tag pin only gives a reproducible build if the rest of the graph is locked too. The ADR didn't account for this.
3. crates.io publishing (ADR step 10, explicitly deferred). .github/workflows/publish-crate.yml runs cargo publish on every v*.*.* tag. cargo publish rejects git dependencies even optional ones, so the moment any ecosystem dep is re-enabled, releases break. Someone must decide: drop publishing, or move these to a registry first.

CI workflow — written but NOT pushed

ADR step 7 asks for a second CI configuration. I wrote .github/workflows/build-features.yml (feature matrix on Rust 1.84 + a job failing the build if any git dep lacks a tag, which would prevent Finding 3 recurring), but the push was rejected: this credential is a PAT without workflow scope. I removed it from the commit rather than work around the restriction. Someone with workflow scope should add it — I can supply the file, or reproduce it from this description.

Note the tag-guard job is the load-bearing part today; while ecosystem = [], both matrix entries compile an identical graph.

[nicholas-ruest]

CI is now live on this PR — first real run results

The workflow scope landed, so .github/workflows/build-features.yml is in this PR and has actually run. Run: https://github.com/LLM-Dev-Ops/incident-manager/actions/runs/30275886700 (conclusion: success).

Job	Result
default (build + test, stable)	pass — 23m50s
ecosystem (build + test, stable)	pass — 23m36s
upstream pins are immutable	pass — 5s
declared MSRV 1.84	fail — non-blocking, on purpose

Getting there surfaced three real defects that were invisible before, none introduced by this branch (its code diff is comments only).

1. The declared rust-version = "1.84" floor does not hold

First run failed both builds in ~30s, before compiling anything:

error: failed to parse manifest at .../chacha20-0.10.1/Cargo.toml
Caused by:
  feature `edition2024` is required
  ...not stabilized in this version of Cargo (1.84.1)

Cargo.lock is gitignored (.gitignore:5), so a clean 1.84 build resolves every transitive dep to its newest release, and several now exceed the floor — chacha20 0.10.1 (edition2024), uuid 1.24.0 (1.85), zeroize 1.9.0 (1.85), wasip2/wit-bindgen (1.85/1.87).

async-lock = "=3.3.0" and base64ct = "=1.6.0" are pinned, per their own comments, solely to hold the 1.84 line. They cannot: two exact pins don't constrain a graph otherwise free to float. ADR-0001's "Toolchain constraint" treats 1.84 as an acceptance criterion for admitting upstream crates — it is already violated with no upstream crate admitted.

The build matrix therefore runs on stable, and a separate non-blocking msrv job keeps the 1.84 failure visible with the evidence inline. Quietly moving everything to stable would have made the symptom vanish while leaving rust-version a false claim. Fix is to commit a lockfile (then add --locked and make msrv blocking) or raise rust-version to the truth — a maintainer decision.

2. rdkafka needs libcurl headers

fatal error: curl/curl.h: No such file or directory. librdkafka builds from source; libcurl4-openssl-dev is now in the apt list. Full set needed for a clean build: cmake protobuf-compiler libssl-dev zlib1g-dev libsasl2-dev libcurl4-openssl-dev.

3. cargo test --workspace has 29 failures and then hangs

• 29 tests FAIL, concentrated in metrics::collectors, metrics::decorators, metrics::interceptors — the clustering points at a shared global metrics registry mutated concurrently (pass alone, fail in parallel).
• The suite then hangs on a test blocking on Redis/NATS/Kafka sockets. The step hit its 20-minute cap in CI, matching ~30 min at 0% CPU locally.

The test step is continue-on-error with a 20-minute cap. Uncapped it burns the full 6h job timeout every push; blocking, it would gate every PR on defects this branch didn't cause. cargo build stays blocking and passes in both configurations — the signal ADR-0001 step 7 actually asks for.

On the pin guard

upstream pins are immutable passes in 5s and is the load-bearing job today. It strips comment lines before matching — the manifest documents the forbidden branch and bare-git forms in prose, and without stripping the guard trips over its own documentation (verified it did, then verified the fixed version still fails on a real branch ref and a real untagged git dep).

While ecosystem = [], the two matrix entries compile an identical graph, so the ecosystem job proves only that the feature name survives. It becomes a real second configuration the moment a dep: entry lands.

[nicholas-ruest]

Update — sentinel is unblocked at v0.1.1; pinned for real (0b473db)

The sentinel section of the PR description above is now out of date. Correcting it here rather than silently editing it.

When this PR was opened, sentinel's only tag was v0.1.0, which I rejected for two independent defects. A new tag v0.1.1 has since been cut. I re-checked it instead of assuming — the working assumption going in was that v0.1.1 was just a tag on the same content and would still fail. That assumption was wrong: both defects are genuinely fixed.

What changed between v0.1.0 and v0.1.1

Defect at v0.1.0	Status at v0.1.1
Crate named sentinel-core, needing a package = "sentinel-core" alias	Fixed — crates/sentinel-core/Cargo.toml declares name = "llm-sentinel-core", no alias needed
sentinel-api required axum-test = "^16.5" (yanked; workspace wouldn't even resolve)	Fixed — now axum-test = "18.2"; workspace resolves
sentinel-core E0599 no method named validate_length found for struct ServiceId against validator 0.18	Fixed — events.rs no longer applies #[validate(length(...))] to the ServiceId newtype

Note v0.1.1 is not merely a re-tag: git diff --stat v0.1.0 v0.1.1 is 211 files / +49416 lines.

Real build output

Clean clone of tag v0.1.1 (remote tag → commit 504b0ea), built standalone:

cargo build --workspace
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 14m 52s
    EXIT=0                      (warnings only; zero errors)

In this repo, after pinning:

cargo build --workspace --features ecosystem
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 7m 32s   EXIT=0
cargo build --workspace
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 2m 03s   EXIT=0

cargo tree | grep -c 'llm-sentinel-core|llm-observatory|llm-governance|llm-analytics|policy-engine'
  -> 0        (ADR check 3: default graph still unchanged)
cargo tree --features ecosystem | grep llm-sentinel-core
  -> llm-sentinel-core v0.1.0 (https://github.com/LLM-Dev-Ops/sentinel?tag=v0.1.1#504b0ea4)

What this PR now does differently

llm-sentinel-core = { git = "https://github.com/LLM-Dev-Ops/sentinel", tag = "v0.1.1", optional = true }
ecosystem = ["dep:llm-sentinel-core"]

ecosystem is no longer a no-op. ADR-0001 Finding 1 is resolved for one of the five members — --features ecosystem finally compiles a different crate graph than the default build. The pinning CI guard still passes (the new line carries an immutable tag, no branch =).

Revised scorecard deltas

#	Check	Was	Now
2	every git = carries tag =	PASS (vacuously)	PASS (for real — one git dep, tag-pinned)
4	--features ecosystem pulls upstream crates	FAIL	PARTIAL — 1 of 4 runtime crates (sentinel) now pulled
6	cfg(feature = "ecosystem") gates in all three adapters	FAIL	still FAIL — see caveat 3

Caveats — stated, not glossed over

1. MSRV unverified for sentinel. The standalone build ran on stable 1.97.1, the only toolchain installed in my environment. ADR-0001 step 1 specifies Rust 1.84, which I could not install here. So sentinel's 1.84 compliance is asserted-but-unverified — the same gap the msrv job already tracks for this repo's own floor. CI on 1.84 is the real check.
2. Upstream version not bumped. sentinel left workspace.package.version at 0.1.0 when cutting v0.1.1, so cargo tree prints llm-sentinel-core v0.1.0 (...?tag=v0.1.1#...). Cosmetic; the tag is what pins us.
3. from_sentinel is still commented out. Restoring it needs the conversion written against v0.1.1's actual AnomalyEvent shape and gated on ecosystem. I left it commented rather than guessing at a shape I hadn't compiled — enabling the feature today pulls the crate in but changes no behaviour.

Still needing a human decision

• cargo publish now genuinely breaks on the next release. This was flagged as hypothetical in the original description; it is live as of this commit. .github/workflows/publish-crate.yml runs cargo publish on every v*.*.* tag, and cargo publish rejects git dependencies even optional ones. Someone must either drop that workflow, or move llm-sentinel-core to a registry, before the next tag on this repo.
• The other four upstreams (observatory, governance-dashboard, analytics-hub, policy-engine) still have 0 tags and remain disabled. Unchanged.