From 8b431ad275295eedd2fe4f14f30f721e8b160d8f Mon Sep 17 00:00:00 2001 From: Julien CHABLE Date: Fri, 21 Aug 2026 21:02:04 +0200 Subject: [PATCH 01/19] docs(audit): design spec for the `okf audit` corpus-level query Adds the design for a new CLI verb, core query type and agent tool that answer "which concepts are past stale_after and were never verified by a human?" across a whole bundle -- the question OKF v0.2 makes askable but ships no query surface for. Three units: ConceptAudit (core, shared computation), the `okf audit` verb (text + --json), and the read-only okf_audit agent/MCP tool. No new frontmatter fields: everything derives from v0.2 sections 5.3-5.5. Co-Authored-By: Claude Opus 5 (1M context) --- .../specs/2026-08-21-okf-audit-design.md | 515 ++++++++++++++++++ 1 file changed, 515 insertions(+) create mode 100644 docs/superpowers/specs/2026-08-21-okf-audit-design.md diff --git a/docs/superpowers/specs/2026-08-21-okf-audit-design.md b/docs/superpowers/specs/2026-08-21-okf-audit-design.md new file mode 100644 index 0000000..589a10e --- /dev/null +++ b/docs/superpowers/specs/2026-08-21-okf-audit-design.md @@ -0,0 +1,515 @@ +# Design — `okf audit` : requête corpus-level sur les signaux de confiance et de fraîcheur + +Date : 2026-08-21 +Statut : validé, prêt pour le plan d'implémentation + +## 1. Objectif + +Répondre, sur un bundle entier et en une commande, à la question que l'article +[« OKF v0.2 Quietly Admits the Folder Has a Ceiling »](https://medium.com/@davidroliver/okf-v0-2-quietly-admits-the-folder-has-a-ceiling-the-way-up-is-a-library-25fa54e872f9) +(David R Oliver, 2026-08-01) met au centre : + +> *which of my concepts are past their `stale_after` date and have never been verified by a human?* + +L'argument : v0.2 a livré le **schéma** d'une base de connaissances (provenance, +trust, lifecycle) en déclarant l'infrastructure de requête hors périmètre. Les +champs `generated`/`verified`/`status`/`stale_after` ne rapportent rien tant +qu'on ne peut pas les interroger **à l'échelle du corpus** ; les ouvrir un par un +marche à 40 concepts, coûte cher à 4 000, et devient une requête de base de +données déguisée en YAML à 40 000. + +OKF4net a déjà tout le socle sémantique — `Trust.DeriveTier` (§5.3), +`Lifecycle`/`IsStale` (§5.4/§5.5), `Frontmatter.TrustTier` — mais **aucune +surface de requête** : ni les 7 verbes CLI ni les 11 tools agents ne posent de +question corpus-level. `okf audit` est cette surface. + +### 1.1 Pourquoi pas `okf validate` + +`BundleValidator` émet déjà un `ConceptStale` (warning) par concept périmé +([Validate.cs:404](../../../src/OKF4net/Validate.cs)). La séparation est +délibérée et doit le rester : + +| | `validate` | `audit` | +|---|---|---| +| Question | ce bundle est-il conforme §11 ? | quels concepts demandent une action ? | +| Sortie | diagnostics par fichier, sévérités | compteurs corpus + worklist filtrable | +| Filtrable | non | oui (`--stale`, `--trust`, `--status`, `--type`) | +| Code retour | 1 si non conforme | **toujours 0** (un audit rapporte, il ne juge pas) | +| Sorties figées | goldens byte-exact v0.1 | nouveaux goldens (§7.3) | + +Corollaire dur : **aucune sortie existante de `validate`/`info` ne change**. Les +goldens v0.1 restent intacts, ce qui exclut d'emblée l'alternative « ajouter des +diagnostics `Info` au validateur » (§10, alternative B). + +## 2. Périmètre + +**Dans le périmètre** — trois unités, une par couche : + +1. `ConceptAudit` : le calcul, dans le cœur, partagé. +2. Le verbe CLI `okf audit`, texte + `--json`. +3. Le tool agent/MCP `okf_audit`, lecture seule. + +**Hors périmètre** (au backlog, mémorisé, non traité ici) : audit fédéré +multi-bundles via `OKF4net.Catalog` ; liens typés `links:` et export +property-graph ; `--fail-on` pour gater la CI ; recherche par passages/BM25 ; +tout champ frontmatter hors v0.2. + +## 3. Unité 1 — `ConceptAudit` (cœur) + +Nouveau fichier `src/OKF4net/Audit.cs`, zéro dépendance, calqué sur le doublet +existant `BundleValidator.Validate(bundle, clock) → ValidationReport`. + +### 3.1 API publique + +```csharp +/// Les prédicats de sélection d'un audit (§5.3–§5.5). Combinés en ET ; `default` ne filtre rien. +public readonly record struct AuditQuery( + bool StaleOnly = false, + IReadOnlySet? Trust = null, + ConceptStatus? Status = null, + string? Type = null) +{ + /// La requête qui retient tous les concepts. + public static AuditQuery All => default; + + /// Vrai dès qu'au moins un prédicat est posé. + public bool IsFiltered => StaleOnly || Trust is not null || Status is not null || Type is not null; +} + +/// Un concept retenu par un audit, avec ses signaux déjà dérivés. +public readonly record struct AuditFinding( + ConceptId Id, + string Path, + string? Type, + string? Title, + TrustTier Trust, + Lifecycle Lifecycle, + bool IsStale); + +/// Le résultat d'un audit : compteurs sur tout le bundle + les concepts sélectionnés. +public sealed class AuditReport +{ + public DateOnly AsOf { get; } + public int ConceptCount { get; } + public IReadOnlyDictionary TrustCounts { get; } // les 3 clés toujours présentes + public IReadOnlyDictionary StatusCounts { get; } // les 3 clés toujours présentes + public int StaleCount { get; } + public IReadOnlyList Findings { get; } // trié par Id, ordinal +} + +/// Interroge un bundle sur ses signaux §5.3–§5.5. Ne lit rien sur disque, n'écrit rien, ne lève rien. +public static class ConceptAudit +{ + public static AuditReport Run(Bundle bundle, AuditQuery query = default, IOkfClock? clock = null); +} +``` + +### 3.2 Sémantique, point par point + +- **Horloge.** `clock ?? new SystemClock()`, `AsOf = clock.Today`, exactement + comme `BundleValidator.Validate`. Aucun `DateTime.UtcNow` enfoui : c'est ce qui + rend les tests et les goldens déterministes (`FixedClock` existe déjà côté + tests). +- **Périmètre des compteurs.** `TrustCounts`, `StatusCounts`, `StaleCount` et + `ConceptCount` portent **toujours sur le bundle entier**, jamais sur la + sélection. Le dénominateur reste stable quand on filtre ; `Findings` seul + bouge. Les trois clés de chaque dictionnaire sont toujours présentes (valeur 0 + incluse) pour que la sortie ait une forme fixe. +- **Périmètre des concepts.** `bundle.Concepts` uniquement. Les `index.md` et + `log.md` (§8/§9) sont exposés séparément par `Bundle` et ne sont pas des + concepts ; les fichiers illisibles restent dans `bundle.ParseErrors` (chargement + permissif §3) et n'entrent dans aucun compteur. +- **Tier de confiance.** `concept.Document.Frontmatter.TrustTier`, donc + `Trust.DeriveTier` (§5.3) : un `human:` ⇒ `HumanReviewed`, sinon tout + vérificateur ⇒ `MachineConfirmed`, liste vide ⇒ `Unverified`. +- **Staleness.** `Lifecycle.IsStale(AsOf)`, donc §5.5 : `AsOf >= stale_after`. + Une borne exacte (`AsOf == stale_after`) est **périmée**. +- **`stale_after` malformé** ⇒ `IsStale` faux, jamais dans la worklist. C'est + `validate` qui possède le diagnostic `StaleAfterInvalid` ; l'audit ne le + redouble pas. +- **Statut inconnu** (`status: retired` dans les fixtures) ⇒ compté comme + `Stable`, conformément à §5.4 (« absent ou inconnu ⇒ stable »). `Lifecycle` + transporte `StatusIsKnown`, mais l'audit ne l'expose pas : signaler la valeur + inconnue est le travail de `validate`. +- **`Type`** : comparaison **ordinale exacte** sur `Frontmatter.Type`. Pas de + repli de casse — un concept sans `type` n'est jamais retenu par `--type`. La + question « `BigQuery Table` et `bigquery-table` sont-ils le même label ? » + appartient au futur lint de vocabulaire (backlog), pas ici. +- **Tri.** `Findings` trié par `ConceptId` ordinal croissant. Déterministe, et + indépendant de l'ordre de parcours du système de fichiers. +- **Robustesse.** Aucune exception : errors-as-data comme le reste du cœur. Un + bundle vide donne des compteurs à zéro et `Findings` vide. + +## 4. Unité 2 — verbe CLI `okf audit` + +Dans `src/OKF4net.Cli/OkfCli.cs` : `CmdAudit(string[] args, TextWriter stdout)`, +branché dans le `switch` de `Run`, plus `JsonOutput.WriteAudit`. + +### 4.1 Grammaire + +``` +okf audit [--stale] [--trust ] [--status ] [--type ] + [--as-of ] [--json] +``` + +| Flag | Valeur | Effet | +|---|---|---| +| `--stale` | — | ne retient que les concepts périmés à la date d'observation | +| `--trust` | liste séparée par `,` parmi `unverified`, `machine-confirmed`, `human-reviewed` | ne retient que ces tiers | +| `--status` | `draft` \| `stable` \| `deprecated` | ne retient que ce statut | +| `--type` | chaîne | ne retient que ce `type` (exact, ordinal) | +| `--as-of` | `YYYY-MM-DD` | fixe la date d'observation (défaut : aujourd'hui, UTC) | +| `--json` | — | document JSON unique, ligne terminée | + +Les prédicats se combinent en **ET**. La question de l'article s'écrit : + +```sh +okf audit bundles/acme_retail --stale --trust unverified,machine-confirmed +``` + +**Un seul vocabulaire** partout (entrée CLI, texte, JSON, tool agent) : +`unverified` / `machine-confirmed` / `human-reviewed` et +`draft` / `stable` / `deprecated`. Pas d'alias, pas de raccourci `--unverified` — +un booléen mentirait sur le cas `machine-confirmed`, qui est aussi « jamais +relu par un humain ». + +Règles de parsing des valeurs, pour lever toute ambiguïté : + +- `--trust` et `--status` sont des **vocabulaires** : chaque entrée est trimée + puis validée, les doublons sont absorbés (`IReadOnlySet`), une entrée vide + (`--trust "a,,b"` ou `--trust ""`) est rejetée avec le message « unknown trust + tier » et la valeur fautive citée. +- `--type` est une **valeur libre** : prise verbatim, sans trim ni repli de + casse, puisque le spec ne contraint pas le vocabulaire de `type`. +- Flag répété : la **première occurrence gagne**, comportement hérité de + `FlagValue` (`Array.IndexOf`) et commun à tous les verbes existants. + +**Piège de parsing à ne pas rater.** `--trust`, `--status`, `--type` et `--as-of` +consomment le token suivant : ils doivent être déclarés dans les `valuedFlags` de +`Positional(args, "", ...)`, sinon `okf audit --as-of 2099-06-01 mon/bundle` +prendrait `2099-06-01` pour le chemin du bundle. C'est exactement ce que `--out` +a résolu pour `render` ([OkfCli.cs:130](../../../src/OKF4net.Cli/OkfCli.cs#L130)). + +### 4.2 Deux modes de présentation, une seule sélection + +**Sans aucun flag de filtre**, `okf audit ` sélectionne le **même +ensemble** que `--stale` ; seule la présentation change. Cette équivalence est +une règle testable, pas un effet de bord. + +Concrètement, le CLI passe `new AuditQuery(StaleOnly: true)` dans les deux cas : +`AuditQuery.All` (aucun prédicat) n'est jamais utilisé par le verbe — il ne sert +qu'aux appelants qui veulent la totalité du corpus, dont le tool agent invoqué +avec `stale: false` et aucun autre filtre. + +**Mode rapport** (aucun flag de filtre) — synthèse + worklist : + +``` +bundle: tests/fixtures/okf_v02 +as of: 2099-06-01 +concepts: 2 + +trust: + 1 human-reviewed + 0 machine-confirmed + 1 unverified + +status: + 0 draft + 2 stable + 0 deprecated + +stale: 1 of 2 past stale_after + +needs attention (1): + metrics/dau stale 2099-01-01 human-reviewed stable +``` + +Conventions d'alignement reprises telles quelles de `info` : libellé + padding +jusqu'à la colonne 13 (`bundle:` suivi de 5 espaces), compteurs en ` {n,4} {label}`. +Ordre des tiers : du plus fort au plus faible. Ordre des statuts : l'ordre du +cycle de vie §5.4 (`draft`, `stable`, `deprecated`), pas l'ordre alphabétique. +Worklist vide ⇒ la dernière section devient la ligne `needs attention: none`. +Aucun plafond : c'est un CLI, la sortie se pipe. + +**Mode requête** (au moins un flag de filtre) — une ligne par concept, rien +d'autre, pour rester pipe-friendly (`| wc -l`, `| xargs`) : + +``` +metrics/dau stale 2099-01-01 human-reviewed stable +``` + +Sélection vide ⇒ **aucune sortie**, code 0. + +**Format d'une ligne de concept** (identique dans les deux modes, à l'indentation +de deux espaces près en mode rapport) — quatre champs séparés par deux espaces : + +1. l'id du concept (`ConceptId`, donc toujours normalisé avec `/`) ; +2. la fraîcheur : `stale `, `fresh `, ou + `no-stale-after` si le champ est absent **ou malformé** ; +3. le tier de confiance ; +4. le statut résolu. + +Asymétrie assumée entre texte et JSON sur le cas malformé : le texte affiche +`no-stale-after` (un `stale_after` illisible ne dit rien de la fraîcheur, et +c'est `validate` qui signale la valeur fautive), tandis que le JSON conserve la +valeur brute dans `staleAfter` avec `stale: false` — un outil qui consomme le +JSON doit pouvoir distinguer « champ absent » de « champ present mais illisible » +sans relire les fichiers. + +Imprimer l'id plutôt que le chemin n'est pas cosmétique : `ConceptId.FromPath` +normalise toujours en `/`, donc les goldens de `audit` sont comparables +byte-for-byte sur les trois OS **sans** la normalisation `Replace('\\','/')` que +`validate.out` doit subir ([GoldenParityTests.cs:74-85](../../../tests/OKF4net.Tests/GoldenParityTests.cs#L74-L85)). + +### 4.3 Codes de retour et erreurs + +- **0** : succès, y compris avec des findings. `audit` rapporte, il ne juge pas + la conformité. Pas de `--fail-on` tant qu'un utilisateur ne le demande pas. +- **1** : erreur d'invocation ou bundle illisible, via `CliOperationException`, + rendue par `Run` en `error: {message}` sur stderr. + +Messages exacts (nouveaux) : + +| Cas | stderr | +|---|---| +| `--as-of` invalide | `error: --as-of is not a valid YYYY-MM-DD date: "2026-13-01"` | +| `--trust` inconnu | `error: unknown trust tier "foo"; expected unverified, machine-confirmed or human-reviewed` | +| `--status` inconnu | `error: unknown status "foo"; expected draft, stable or deprecated` | + +Réutilisés tels quels : `error: missing ` (`Positional`) et +`error: --as-of requires a value` (`FlagValue`). + +`--as-of` est parsé avec `DateOnly.TryParseExact("yyyy-MM-dd", CultureInfo.InvariantCulture)` +— même contrat que `Lifecycle.From`, et compatible `InvariantGlobalization` (AOT). + +### 4.4 Texte d'aide + +Ajouter à `Usage` la ligne du verbe, alignée sur les autres : + +``` + audit Report trust, freshness and lifecycle across the bundle +``` + +et étendre la ligne d'option existante en +`--json Machine-readable output for validate/info/audit`. Le commentaire +de classe de `OkfCli` (« Seven subcommands ») passe à huit et cite `audit`. + +## 5. Unité 3 — tool agent `okf_audit` + +Dans `OkfBundleTools`, sur le modèle de `Search` : + +```csharp +[Description("Audit the bundle's trust, freshness and lifecycle signals: counts by trust tier and status, plus the concepts needing attention. Filter with stale/trust/status/type.")] +public string Audit( + [Description("Only concepts past their stale_after date. Defaults to true.")] bool stale = true, + [Description("Comma-separated trust tiers to include: unverified, machine-confirmed, human-reviewed.")] string? trust = null, + [Description("Only concepts with this lifecycle status: draft, stable or deprecated.")] string? status = null, + [Description("Only concepts with this frontmatter type (exact match).")] string? type = null) +``` + +Enregistré dans `GetTools()` via `AIFunctionFactory.Create(Audit, "okf_audit")`. +**Lecture seule**, donc absent de `WriteToolNames` : il reste disponible quand +`okf-mcp` tourne en mode read-only. + +Différences assumées avec le CLI : + +- rend **toujours** la forme rapport (synthèse + liste), même filtré : la + synthèse est du contexte utile pour un agent ; +- **plafonne à 20 findings**, suivis de `… and N more (narrow with stale/trust/status/type)` + — même plafond que `okf_search`, et c'est précisément l'économie de contexte + que l'article défend ; +- omet la ligne `bundle:` (le tool est lié à un seul bundle) ; +- valeurs invalides de `trust`/`status` ⇒ message d'usage rendu comme chaîne (pas + d'exception), sur le modèle de `SearchUsageMessage`. + +**Le rendu texte n'est pas partagé entre CLI et Agents** : seul le calcul +(`ConceptAudit`) l'est. Raison : les octets du CLI sont verrouillés par des +goldens et ne doivent jamais bouger parce qu'une chaîne destinée à un agent a +changé ; les deux rendus font une quinzaine de lignes chacun. C'est exactement le +partage retenu pour `ConceptSearch` (scorer commun, présentations distinctes). + +## 6. Sortie `--json` + +`System.Text.Json` **source-generated** — obligatoire, le CLI est publié Native +AOT. Nouveaux records internes dans `JsonOutput.cs`, plus +`[JsonSerializable(typeof(AuditJsonResult))]` sur `CliJsonContext` (le générateur +couvre le graphe atteignable ; seul le type racine s'annote). Nommage camelCase +via la policy déjà en place. + +```csharp +internal sealed record AuditQueryJson(bool Stale, IReadOnlyList? Trust, string? Status, string? Type); +internal sealed record TrustCountsJson(int HumanReviewed, int MachineConfirmed, int Unverified); +internal sealed record StatusCountsJson(int Draft, int Stable, int Deprecated); +internal sealed record AuditFindingJson(string ConceptId, string Path, string? Type, string? Title, string Trust, string Status, string? StaleAfter, bool Stale); +internal sealed record AuditJsonResult( + string Bundle, string AsOf, int ConceptCount, AuditQueryJson Query, + TrustCountsJson Trust, StatusCountsJson Status, int StaleCount, + IReadOnlyList Findings); +``` + +Exemple (`okf audit tests/fixtures/okf_v02 --as-of 2099-06-01 --json`, ici +ré-indenté ; la sortie réelle est une seule ligne suivie de `\n`) : + +```json +{ + "bundle": "tests/fixtures/okf_v02", + "asOf": "2099-06-01", + "conceptCount": 2, + "query": { "stale": true, "trust": null, "status": null, "type": null }, + "trust": { "humanReviewed": 1, "machineConfirmed": 0, "unverified": 1 }, + "status": { "draft": 0, "stable": 2, "deprecated": 0 }, + "staleCount": 1, + "findings": [ + { + "conceptId": "metrics/dau", + "path": "tests/fixtures/okf_v02/metrics/dau.md", + "type": "Metric", + "title": "Daily Active Users", + "trust": "human-reviewed", + "status": "stable", + "staleAfter": "2099-01-01", + "stale": true + } + ] +} +``` + +Décisions de schéma : + +- `--json` rend **toujours** le document complet, dans les deux modes : les + machines voient une seule forme, quelle que soit la présentation texte. +- `query` **rejoue la requête appliquée**, ce qui rend le document + auto-descriptif et lève l'ambiguïté du mode par défaut (`stale: true` sans + qu'aucun flag n'ait été passé). +- `staleAfter` porte la valeur **brute** du frontmatter ; elle vaut `null` si le + champ est absent, et la valeur brute non parsable s'il est malformé (auquel cas + `stale` est `false`). +- Pas de champ `statusKnown` : le statut inconnu est le domaine de `validate` + (§3.2). Schéma minimal, donc stable. +- `path` est le chemin réel du concept, donc porteur du séparateur natif de l'OS + — c'est le seul champ à normaliser côté sortie C# dans le test golden (§7.3). + +## 7. Tests + +### 7.1 Unitaires — `tests/OKF4net.Tests/AuditTests.cs` (nouveau) + +Bundles synthétiques + `FixedClock` (déjà présent). + +1. Les trois tiers sont comptés distinctement (`human:` ⇒ human-reviewed ; + vérificateur non-humain seul ⇒ machine-confirmed ; absence ⇒ unverified). +2. Statut inconnu compté comme `stable`. +3. Borne de staleness §5.5 : `AsOf == stale_after` ⇒ périmé ; `AsOf == stale_after - 1j` ⇒ non. +4. `stale_after` malformé ⇒ non périmé, absent de la worklist. +5. `stale_after` absent ⇒ jamais périmé. +6. Les prédicats se combinent en ET (`--stale` + `--trust` ne retient que + l'intersection). +7. `Findings` trié par id ordinal, indépendamment de l'ordre de chargement. +8. Les compteurs portent sur tout le bundle même quand la requête filtre. +9. Bundle vide ⇒ compteurs à zéro, `Findings` vide, aucune exception. +10. Fichiers illisibles (`ParseErrors`) exclus des compteurs, sans exception. +11. `clock: null` ⇒ `AsOf` = date UTC du jour. +12. `--type` : match ordinal exact ; casse différente ⇒ pas de match ; concept + sans `type` ⇒ jamais retenu. + +### 7.2 CLI — `tests/OKF4net.Tests/CliTests.cs` (existant) + +13. Mode rapport : sections et alignements attendus. +14. Mode requête : uniquement des lignes de concepts, pas de synthèse. +15. **Équivalence** : `audit ` et `audit --stale` sélectionnent le même + ensemble (comparaison sur les ids). +16. `--json` : document parsable, champs et valeurs attendus, `query` rejoué. +17. `--as-of` invalide ⇒ code 1 + stderr exact. +18. `--trust` inconnu ⇒ code 1 + stderr exact. +19. `--status` inconnu ⇒ code 1 + stderr exact. +20. `--trust` avec une entrée vide (`unverified,,human-reviewed`) ⇒ code 1 ; + avec un doublon (`unverified,unverified`) ⇒ code 0 et même résultat qu'une + seule occurrence. +21. **Régression de parsing** : `okf audit --as-of ` (flags avant + le positionnel) résout le bon bundle ; idem pour `--trust`, `--status`, `--type`. +22. Code 0 malgré des findings. +23. Sélection vide ⇒ sortie vide, code 0. +24. `--help` liste `audit`. + +### 7.3 Goldens — `tests/OKF4net.Tests/GoldenParityTests.cs` + +Bundle réutilisé : `tests/fixtures/okf_v02` (2 concepts, déjà porteur des champs +v0.2), avec `--as-of 2099-06-01` **figé** — cette date rend `metrics/dau` +(`stale_after: 2099-01-01`) périmé sans qu'aucune fixture ne soit créée ni +modifiée. Aucun nouveau bundle de fixtures. + +Nouveaux fichiers dans `tests/fixtures/golden/` : `audit-v02.out`, +`audit-v02.exitcode`, `audit-v02.json`. + +- `audit-v02.out` : comparé **byte-for-byte sans normalisation** (la sortie ne + contient que des ids de concepts, toujours en `/`) — sauf la ligne `bundle:`, + qui reprend l'argument tel que passé, d'où le recours à `WithRepoRootAsCwd` + comme pour `validate`/`info`. +- `audit-v02.json` : comparé après `Replace('\\','/')` **sur la sortie C#, + jamais sur le golden**, à cause du champ `path` (§6) — même traitement, et même + justification, que `validate.out`. + +Provenance à consigner explicitement, la règle du repo l'exige : ces goldens sont +**écrits à la main** et vérifiés contre le texte du spec (§5.3 tiers, §5.4 +statuts, §5.5 staleness), **pas** capturés depuis le CLI de référence — `audit` +n'existe pas en amont. À documenter dans `tests/fixtures/README.md` et dans le +commentaire de classe de `GoldenParityTests`, au même titre que +`validate-v02.out`. + +### 7.4 Non-régression + +Aucun golden existant ne change. `dotnet test OKF4net.sln` doit rester vert sans +qu'un seul fichier de `tests/fixtures/` préexistant soit touché. + +### 7.5 Agents — `tests/OKF4net.Tests/Agents/` + +25. `okf_audit` est enregistré dans `GetTools()`. +26. Il est **absent** de `WriteToolNames` (donc exposé en mode read-only). +27. Plafond à 20 findings + ligne `… and N more`. +28. `trust`/`status` invalides ⇒ message d'usage rendu, pas d'exception. + +## 8. Documentation à mettre à jour + +- `README.md` : liste des verbes CLI, et la table §-du-spec → type (ajouter + `ConceptAudit` en regard de §5.3–§5.5). +- `CHANGELOG.md` : entrée sous `Unreleased`. +- `ROADMAP.md` : inscrire l'échelle de l'article (index → recherche → graphe → + bibliothèque fédérée) et marquer ce premier barreau. +- `CLAUDE.md` : une ligne sur `ConceptAudit` comme surface de requête unique + partagée CLI/Agents — même statut que la note « ne pas forker `ConceptSearch` ». + +## 9. Contraintes respectées + +- **Zéro dépendance tierce** : `Audit.cs` n'utilise que la BCL ; aucun + `PackageReference` ajouté nulle part. +- **Native AOT** : JSON source-generated obligatoire, parsing de date en + `InvariantCulture` (compatible `InvariantGlobalization`). +- **Fidélité au spec** : aucun champ frontmatter nouveau, aucune extension. Tout + se dérive de champs v0.2 existants (§5.3/§5.4/§5.5). `audit` est un + **consommateur** du spec, pas une extension de celui-ci. +- **Fixtures** : aucune fixture existante modifiée ; les nouvelles relèvent de + l'exception documentée « comportement non couvert par une capture existante », + vérifiées à la main contre le texte du spec. +- **Conventions de fichier** : en-tête `// SPDX-License-Identifier: LGPL-3.0-or-later`, + namespace file-scoped, XML doc sur toute l'API publique, nullable activé, + warnings = erreurs. + +## 10. Alternatives écartées + +**A. Calculer dans le CLI, dupliquer côté Agents plus tard.** Écartée : elle +forke la logique dans deux couches, ce que le repo interdit explicitement pour le +scorer de recherche. Une seule implémentation, deux présentations. + +**B. Étendre `BundleValidator` avec des diagnostics `Info`, `audit` devenant une +vue filtrée de `validate`.** Écartée pour deux raisons cumulatives : elle mélange +la conformité §11 avec de l'hygiène éditoriale, et elle modifierait la sortie de +`validate`, donc les goldens byte-exact — interdit. + +**C. Un flag `--unverified` plutôt que `--trust `.** Écartée : « jamais +relu par un humain » recouvre deux tiers (`unverified` **et** +`machine-confirmed`) ; un booléen serait faux sur le second, et il ferait un +second vocabulaire en plus des noms de tiers. + +**D. Code de retour non nul quand il y a des findings.** Reportée : un concept +périmé n'est pas une erreur de conformité, et l'usage décrit par l'article est +une worklist hebdomadaire routée par équipe, pas un gate de CI. Si le besoin de +gate apparaît, il se traitera par un `--fail-on` explicite. From 6bf36b507299e40455a85af8fdda26e09e516274 Mon Sep 17 00:00:00 2001 From: Julien CHABLE Date: Fri, 21 Aug 2026 21:10:24 +0200 Subject: [PATCH 02/19] docs(audit): tighten the audit spec after a close re-read Seven defects found by re-reading the spec against the code: - AuditQuery is a record struct whose generated equality compares the trust set by reference; the type advertised a value equality it does not have. Documented as a remark, not relied on. - No way to select the whole corpus: without filters the verb reports the stale worklist, so --json findings are not the corpus. Documented the three-tier idiom and why no --all is added. - audit-v02.exitcode would be a golden for a constant (exit is always 0); dropped, asserted inline like Info_output_matches_golden. - The set of mode-switching flags was never enumerated: --as-of and --json must not switch to query mode, which the golden depends on. - query.trust JSON ordering was unspecified; pinned to ladder order so the document is reproducible with several tiers. - The position of the audit line in the usage text was unpinned while test 24 checks it; pinned right after validate. - CLAUDE.md and ROADMAP.md are concurrently modified by another session: noted as end-of-branch work to avoid a rebase conflict. Co-Authored-By: Claude Opus 5 (1M context) --- .../specs/2026-08-21-okf-audit-design.md | 56 +++++++++++++++++-- 1 file changed, 50 insertions(+), 6 deletions(-) diff --git a/docs/superpowers/specs/2026-08-21-okf-audit-design.md b/docs/superpowers/specs/2026-08-21-okf-audit-design.md index 589a10e..44426ef 100644 --- a/docs/superpowers/specs/2026-08-21-okf-audit-design.md +++ b/docs/superpowers/specs/2026-08-21-okf-audit-design.md @@ -26,7 +26,7 @@ question corpus-level. `okf audit` est cette surface. ### 1.1 Pourquoi pas `okf validate` `BundleValidator` émet déjà un `ConceptStale` (warning) par concept périmé -([Validate.cs:404](../../../src/OKF4net/Validate.cs)). La séparation est +([Validate.cs:404](../../../src/OKF4net/Validate.cs#L404)). La séparation est délibérée et doit le rester : | | `validate` | `audit` | @@ -63,6 +63,13 @@ existant `BundleValidator.Validate(bundle, clock) → ValidationReport`. ```csharp /// Les prédicats de sélection d'un audit (§5.3–§5.5). Combinés en ET ; `default` ne filtre rien. +/// +/// L'égalité générée compare Trust par référence (c'est le comportement de +/// EqualityComparer<IReadOnlySet<T>>.Default) : deux requêtes logiquement +/// identiques peuvent être inégales. Ne pas s'appuyer dessus, ni utiliser une +/// AuditQuery comme clé de dictionnaire. Le type reste un record struct pour +/// `with` et `ToString`, pas pour son égalité. +/// public readonly record struct AuditQuery( bool StaleOnly = false, IReadOnlySet? Trust = null, @@ -201,6 +208,21 @@ Concrètement, le CLI passe `new AuditQuery(StaleOnly: true)` dans les deux cas qu'aux appelants qui veulent la totalité du corpus, dont le tool agent invoqué avec `stale: false` et aucun autre filtre. +**Les flags de filtre sont exactement `--stale`, `--trust`, `--status` et +`--type`.** `--as-of` et `--json` n'en font pas partie et ne changent jamais de +mode : `okf audit --as-of 2099-06-01` reste en mode rapport — c'est +précisément l'invocation du golden (§7.3). + +**Conséquence à assumer : `audit` ne sait pas sélectionner tout le corpus en une +option.** Sans filtre il rend la worklist des périmés, et `--json` porte alors +des `findings` limités à ceux-ci — les compteurs, eux, restent corpus-larges. Qui +veut l'inventaire concept par concept énumère les trois tiers +(`--trust unverified,machine-confirmed,human-reviewed`). Aucun `--all` n'est +ajouté : `audit` est une worklist, l'inventaire est déjà le métier de +`okf info --json` et de `okf_browse`. Ce point doit apparaître tel quel dans la +documentation du verbe, faute de quoi un consommateur du JSON prendra `findings` +pour le corpus. + **Mode rapport** (aucun flag de filtre) — synthèse + worklist : ``` @@ -284,7 +306,10 @@ Réutilisés tels quels : `error: missing ` (`Positional`) et ### 4.4 Texte d'aide -Ajouter à `Usage` la ligne du verbe, alignée sur les autres : +Ajouter à `Usage` la ligne du verbe **juste après `validate`** (l'ordre de la +liste est vérifié par le test 24 : conformité d'abord, santé du corpus ensuite), +alignée sur les autres — le verbe occupe 8 colonnes, d'où quatre espaces après +`audit` : ``` audit Report trust, freshness and lifecycle across the bundle @@ -381,6 +406,11 @@ Décisions de schéma : - `query` **rejoue la requête appliquée**, ce qui rend le document auto-descriptif et lève l'ambiguïté du mode par défaut (`stale: true` sans qu'aucun flag n'ait été passé). +- `query.trust` est sérialisé dans **l'ordre du ladder** (`unverified`, + `machine-confirmed`, `human-reviewed`), jamais dans l'ordre de saisie : + `IReadOnlySet` n'a pas d'ordre garanti, et sans cette règle le JSON ne serait + pas reproductible dès qu'on passe plusieurs tiers. `null` quand `--trust` est + absent. - `staleAfter` porte la valeur **brute** du frontmatter ; elle vaut `null` si le champ est absent, et la valeur brute non parsable s'il est malformé (auquel cas `stale` est `false`). @@ -413,8 +443,11 @@ Bundles synthétiques + `FixedClock` (déjà présent). ### 7.2 CLI — `tests/OKF4net.Tests/CliTests.cs` (existant) -13. Mode rapport : sections et alignements attendus. -14. Mode requête : uniquement des lignes de concepts, pas de synthèse. +13. Mode rapport : sections et alignements attendus — **y compris avec `--as-of` + seul**, qui ne doit pas basculer en mode requête (§4.2). +14. Mode requête : uniquement des lignes de concepts, pas de synthèse ; et + l'idiome des trois tiers (`--trust unverified,machine-confirmed,human-reviewed`) + retourne bien la totalité des concepts du bundle. 15. **Équivalence** : `audit ` et `audit --stale` sélectionnent le même ensemble (comparaison sur les ids). 16. `--json` : document parsable, champs et valeurs attendus, `query` rejoué. @@ -437,8 +470,13 @@ v0.2), avec `--as-of 2099-06-01` **figé** — cette date rend `metrics/dau` (`stale_after: 2099-01-01`) périmé sans qu'aucune fixture ne soit créée ni modifiée. Aucun nouveau bundle de fixtures. -Nouveaux fichiers dans `tests/fixtures/golden/` : `audit-v02.out`, -`audit-v02.exitcode`, `audit-v02.json`. +Nouveaux fichiers dans `tests/fixtures/golden/` : `audit-v02.out` et +`audit-v02.json`. **Pas de `audit-v02.exitcode`** : le code de retour d'`audit` +est constamment 0 (§4.3), un golden pour une constante n'apporte rien et grossit +la surface de fixtures. Le test l'assère en ligne (`Assert.Equal(0, r.Code)`), +exactement comme `Info_output_matches_golden`, qui n'a pas non plus de golden de +code retour ; les `*.exitcode` de `validate` n'existent que parce que son code +varie. - `audit-v02.out` : comparé **byte-for-byte sans normalisation** (la sortie ne contient que des ids de concepts, toujours en `/`) — sauf la ligne `bundle:`, @@ -477,6 +515,12 @@ qu'un seul fichier de `tests/fixtures/` préexistant soit touché. - `CLAUDE.md` : une ligne sur `ConceptAudit` comme surface de requête unique partagée CLI/Agents — même statut que la note « ne pas forker `ConceptSearch` ». +**Coordination.** Au moment d'écrire cette spec, `CLAUDE.md` et `ROADMAP.md` +étaient déjà modifiés, non commités, dans le worktree principal par une autre +session (travail viewer/CI). Les toucher ici provoquera un conflit au rebase : +ces deux fichiers sont à traiter en fin de branche, une fois l'autre session +mergée, et non au fil de l'implémentation. + ## 9. Contraintes respectées - **Zéro dépendance tierce** : `Audit.cs` n'utilise que la BCL ; aucun From be3f099ed7d738c5ccebd49322a73ed6fae75ca5 Mon Sep 17 00:00:00 2001 From: Julien CHABLE Date: Fri, 21 Aug 2026 21:14:46 +0200 Subject: [PATCH 03/19] docs(audit): fix five defects found by external review All five verified against the code before accepting: - DateOnly has no (s, format, provider, out) overload, so the parsing contract as written would not compile. Pinned the full five-argument call, matching Lifecycle.From. - --status had two incompatible semantics: scalar in AuditQuery, the JSON and the tool, but described as a deduplicated list in the parsing rules. Settled on scalar; --trust is the only multi-valued flag. - The "unreadable files" test described an impossible state: I/O, permission and non-UTF-8 failures throw BundleLoadException and abort the load, while ParseErrors only collects DocumentParseException and ConceptIdException. Reframed around an invalid-frontmatter document. - The agent tool would have fallen back to SystemClock, bypassing the existing UtcNow/Today seam that ReadConcept and Search already use, making its output depend on the run date. It now passes Today through a private IOkfClock adapter, with a test pinning today == stale_after. - "No fixture created or modified" contradicted the new golden files; stated precisely: no fixture bundle and no existing golden change. Co-Authored-By: Claude Opus 5 (1M context) --- .../specs/2026-08-21-okf-audit-design.md | 71 ++++++++++++++++--- 1 file changed, 60 insertions(+), 11 deletions(-) diff --git a/docs/superpowers/specs/2026-08-21-okf-audit-design.md b/docs/superpowers/specs/2026-08-21-okf-audit-design.md index 44426ef..22710b8 100644 --- a/docs/superpowers/specs/2026-08-21-okf-audit-design.md +++ b/docs/superpowers/specs/2026-08-21-okf-audit-design.md @@ -124,8 +124,13 @@ public static class ConceptAudit incluse) pour que la sortie ait une forme fixe. - **Périmètre des concepts.** `bundle.Concepts` uniquement. Les `index.md` et `log.md` (§8/§9) sont exposés séparément par `Bundle` et ne sont pas des - concepts ; les fichiers illisibles restent dans `bundle.ParseErrors` (chargement - permissif §3) et n'entrent dans aucun compteur. + concepts. Les documents **non parsables** — frontmatter invalide + (`DocumentParseException`) ou clés requises manquantes (`ConceptIdException`) — + sont collectés dans `bundle.ParseErrors` par le chargement permissif (§11) et + n'entrent dans aucun compteur. À ne pas confondre avec un fichier réellement + **illisible** (I/O, droits, contenu non-UTF-8) : `Bundle.Load` lève alors + `BundleLoadException` et abandonne le chargement entier ; le CLI rend cela en + `error:` + code 1 (§4.3), et l'audit ne voit jamais ce cas. - **Tier de confiance.** `concept.Document.Frontmatter.TrustTier`, donc `Trust.DeriveTier` (§5.3) : un `human:` ⇒ `HumanReviewed`, sinon tout vérificateur ⇒ `MachineConfirmed`, liste vide ⇒ `Unverified`. @@ -182,10 +187,15 @@ relu par un humain ». Règles de parsing des valeurs, pour lever toute ambiguïté : -- `--trust` et `--status` sont des **vocabulaires** : chaque entrée est trimée - puis validée, les doublons sont absorbés (`IReadOnlySet`), une entrée vide - (`--trust "a,,b"` ou `--trust ""`) est rejetée avec le message « unknown trust - tier » et la valeur fautive citée. +- `--trust` est le **seul flag à valeurs multiples** : liste séparée par `,`, + chaque entrée trimée puis validée contre le vocabulaire, doublons absorbés + (`IReadOnlySet`), entrée vide (`--trust "a,,b"` ou `--trust ""`) rejetée avec + le message « unknown trust tier » et la valeur fautive citée. +- `--status` prend **une seule valeur**, trimée puis validée contre le + vocabulaire §5.4. Pas de liste : `AuditQuery.Status`, le champ JSON `status` et + le paramètre du tool agent sont tous des scalaires (`ConceptStatus?` / + `string?`). Un besoin multi-statuts se traiterait par un changement de type + cohérent sur les trois surfaces, pas par une tolérance du parseur. - `--type` est une **valeur libre** : prise verbatim, sans trim ni repli de casse, puisque le spec ne contraint pas le vocabulaire de `type`. - Flag répété : la **première occurrence gagne**, comportement hérité de @@ -301,8 +311,16 @@ Messages exacts (nouveaux) : Réutilisés tels quels : `error: missing ` (`Positional`) et `error: --as-of requires a value` (`FlagValue`). -`--as-of` est parsé avec `DateOnly.TryParseExact("yyyy-MM-dd", CultureInfo.InvariantCulture)` -— même contrat que `Lifecycle.From`, et compatible `InvariantGlobalization` (AOT). +`--as-of` est parsé exactement comme `Lifecycle.From` le fait pour `stale_after` +— même contrat, et compatible `InvariantGlobalization` (AOT) : + +```csharp +DateOnly.TryParseExact(raw, "yyyy-MM-dd", CultureInfo.InvariantCulture, DateTimeStyles.None, out var asOf) +``` + +Les cinq arguments sont obligatoires : `DateOnly` n'offre **pas** de surcharge +`(s, format, provider, out)` — seulement `(s, format, out)` et la forme complète +ci-dessus. Omettre `DateTimeStyles.None` ne compile pas. ### 4.4 Texte d'aide @@ -336,6 +354,29 @@ Enregistré dans `GetTools()` via `AIFunctionFactory.Create(Audit, "okf_audit")` **Lecture seule**, donc absent de `WriteToolNames` : il reste disponible quand `okf-mcp` tourne en mode read-only. +**La date d'observation vient de la couture existante, pas d'une horloge neuve.** +Le tool n'expose délibérément pas de paramètre `asOf` (un agent n'a pas à choisir +sa notion d'aujourd'hui), mais il ne doit pas non plus laisser `ConceptAudit` +retomber sur `SystemClock` : `OkfBundleTools` possède déjà le seam interne +`Func UtcNow` et la propriété `Today` qui en dérive — « the shared seam +behind `ReadConcept`'s and `Search`'s staleness checks » +([OkfBundleTools.cs:136](../../../src/OKF4net.Agents/OkfBundleTools.cs#L136)). +`Audit` passe donc `Today` à `ConceptAudit`, via un adaptateur privé de quatre +lignes : + +```csharp +private sealed class PinnedClock(DateOnly today) : IOkfClock +{ + public DateOnly Today { get; } = today; +} +``` + +Sans cela, la sortie du tool dépendrait du jour d'exécution et le cas limite +`today == stale_after` serait intestable. L'alternative — une surcharge publique +`ConceptAudit.Run(bundle, query, DateOnly asOf)` — est écartée pour garder une +seule forme canonique dans le cœur ; l'adaptateur reste local aux Agents et +n'ajoute aucune surface publique. + Différences assumées avec le CLI : - rend **toujours** la forme rapport (synthèse + liste), même filtré : la @@ -436,7 +477,10 @@ Bundles synthétiques + `FixedClock` (déjà présent). 7. `Findings` trié par id ordinal, indépendamment de l'ordre de chargement. 8. Les compteurs portent sur tout le bundle même quand la requête filtre. 9. Bundle vide ⇒ compteurs à zéro, `Findings` vide, aucune exception. -10. Fichiers illisibles (`ParseErrors`) exclus des compteurs, sans exception. +10. Documents non parsables exclus des compteurs, sans exception : un fichier au + frontmatter invalide atterrit dans `ParseErrors`, et `ConceptCount` ne le + compte pas. (Pas de test « fichier illisible » : ce cas lève + `BundleLoadException` au chargement et n'atteint jamais `ConceptAudit`.) 11. `clock: null` ⇒ `AsOf` = date UTC du jour. 12. `--type` : match ordinal exact ; casse différente ⇒ pas de match ; concept sans `type` ⇒ jamais retenu. @@ -467,8 +511,10 @@ Bundles synthétiques + `FixedClock` (déjà présent). Bundle réutilisé : `tests/fixtures/okf_v02` (2 concepts, déjà porteur des champs v0.2), avec `--as-of 2099-06-01` **figé** — cette date rend `metrics/dau` -(`stale_after: 2099-01-01`) périmé sans qu'aucune fixture ne soit créée ni -modifiée. Aucun nouveau bundle de fixtures. +(`stale_after: 2099-01-01`) périmé sans toucher au bundle. Formulation exacte de +l'engagement : **aucun bundle de fixtures n'est créé ni modifié**, et **aucun +golden existant n'est modifié** ; les seuls ajouts sont des goldens neufs, +vérifiés à la main (voir plus bas). Nouveaux fichiers dans `tests/fixtures/golden/` : `audit-v02.out` et `audit-v02.json`. **Pas de `audit-v02.exitcode`** : le code de retour d'`audit` @@ -504,6 +550,9 @@ qu'un seul fichier de `tests/fixtures/` préexistant soit touché. 26. Il est **absent** de `WriteToolNames` (donc exposé en mode read-only). 27. Plafond à 20 findings + ligne `… and N more`. 28. `trust`/`status` invalides ⇒ message d'usage rendu, pas d'exception. +29. **Date pinnée par le seam `UtcNow`** : avec `UtcNow` figé, la sortie est + déterministe, et la borne `today == stale_after` classe bien le concept comme + périmé. Sans ce test, le comportement clé dépendrait du jour d'exécution. ## 8. Documentation à mettre à jour From 9c83a78cd8ebc2d6b630f531a25bfc730adcd7b6 Mon Sep 17 00:00:00 2001 From: Julien CHABLE Date: Fri, 21 Aug 2026 21:59:05 +0200 Subject: [PATCH 04/19] docs(audit): add the implementation plan, and fix the spec's base assumptions The plan covers the spec in six tasks (core query, CLI text modes, JSON, goldens, agent tool, docs), each TDD with real test and implementation code rather than descriptions. Writing it surfaced that the spec was written against the wrong base. FlagValue and Positional's valuedFlags parameter do not exist on origin/main -- both come from the unmerged viewer branch, which added them for `render --out`, and the spec cited that as existing precedent. The plan now opens with Task 0 adding them, copied verbatim from that branch so the eventual merge conflict resolves to identical code, and the spec no longer claims a precedent this base does not have. Same for the subcommand count: six here, not seven. Self-review also caught, in the plan itself: a nullable-narrowing pattern that would not compile under warnings-as-errors, a cap test that would have counted the "concepts:" header as a finding, two unused vocabulary members, and a nested code fence that broke its own block. Co-Authored-By: Claude Opus 5 (1M context) --- .../superpowers/plans/2026-08-21-okf-audit.md | 1721 +++++++++++++++++ .../specs/2026-08-21-okf-audit-design.md | 15 +- 2 files changed, 1733 insertions(+), 3 deletions(-) create mode 100644 docs/superpowers/plans/2026-08-21-okf-audit.md diff --git a/docs/superpowers/plans/2026-08-21-okf-audit.md b/docs/superpowers/plans/2026-08-21-okf-audit.md new file mode 100644 index 0000000..d47e231 --- /dev/null +++ b/docs/superpowers/plans/2026-08-21-okf-audit.md @@ -0,0 +1,1721 @@ +# `okf audit` Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Ajouter une requête corpus-level sur les signaux de confiance et de fraîcheur d'un bundle OKF — « quels concepts ont dépassé `stale_after` sans avoir jamais été vérifiés par un humain ? » — exposée en type de cœur, en verbe CLI et en tool agent. + +**Architecture:** Un seul calcul (`ConceptAudit` dans `src/OKF4net/`), trois consommateurs qui rendent chacun leur propre présentation : le verbe CLI `okf audit` (texte + `--json`), et le tool agent `okf_audit`. Aucun champ frontmatter nouveau : tout dérive de `Frontmatter.TrustTier` (§5.3) et `Frontmatter.Lifecycle` (§5.4/§5.5), déjà présents. + +**Tech Stack:** C# / net10.0, xunit, `System.Text.Json` source-generated (Native AOT), zéro dépendance tierce. + +**Spec:** [docs/superpowers/specs/2026-08-21-okf-audit-design.md](../specs/2026-08-21-okf-audit-design.md) + +## Global Constraints + +- **Zéro dépendance tierce** dans `src/OKF4net/` et `src/OKF4net.Cli/` : BCL uniquement, aucun `PackageReference` ajouté. +- **Tout nouveau fichier source** commence par `// SPDX-License-Identifier: LGPL-3.0-or-later`. +- **Namespaces file-scoped**, XML doc obligatoire sur toute API publique, nullable activé, `TreatWarningsAsErrors` — un warning casse le build. +- **Native AOT** : toute sérialisation JSON passe par `CliJsonContext` (source-generated) ; tout formatage/parsing de date utilise `CultureInfo.InvariantCulture` (le CLI est publié avec `InvariantGlobalization`). +- **Fixtures** : ne jamais modifier un fichier existant sous `tests/fixtures/`. Ce plan n'ajoute que deux goldens neufs, écrits à la main. +- **Aucune sortie existante ne change** : `validate`, `info`, `graph`, `parse`, `fmt`, `render` produisent les mêmes octets qu'avant. Si un golden existant casse, c'est une régression à corriger côté code. +- **Vérification** : `dotnet build OKF4net.sln` (warnings = erreurs) et `dotnet test OKF4net.sln` doivent être verts avant chaque commit. Baseline au départ de cette branche : **912 tests, 0 échec**. +- **Format** : `dotnet format OKF4net.sln` avant le dernier commit (la CI lance `--verify-no-changes`). + +## Base de la branche — à lire avant de commencer + +Cette branche part d'`origin/main`, où le CLI a **six** sous-commandes +(`validate`, `info`, `index`, `graph`, `parse`, `fmt`). Le verbe `render` et le +projet `OKF4net.Viewer` vivent sur la branche `okf-bundle-viewer-static-render`, +non mergée. + +Conséquence concrète : **les deux helpers de parsing que la spec présente comme +existants n'existent pas sur cette base.** `FlagValue` est absent, et +`Positional` n'a pas de paramètre `valuedFlags` — les deux ont été introduits par +la branche viewer pour `render --out`, que la spec §4.1 cite comme précédent. +C'est l'objet de la Task 0. + +Choix retenu : rester sur `origin/main` pour que `audit` soit mergeable +indépendamment du viewer, et **recopier verbatim** l'implémentation de la branche +viewer plutôt que d'en écrire une variante. Les deux branches introduiront donc +le même code ; le conflit au merge se résout en gardant une seule copie, à +l'identique des deux côtés. Ne pas « améliorer » ces deux helpers ici : toute +divergence transformerait une résolution triviale en arbitrage. + +## Écarts assumés par rapport à la spec + +Deux points où ce plan précise la spec plutôt que de la suivre à la lettre — à signaler en revue : + +1. **`AuditVocabulary` (Task 1) n'était pas dans le croquis d'API de la spec.** La spec exige « un seul vocabulaire partout (entrée CLI, texte, JSON, tool agent) ». Sans type partagé, cette exigence se traduirait par des littéraux `"human-reviewed"` dupliqués dans `OKF4net.Cli` et `OKF4net.Agents`, libres de diverger sans qu'aucun test ne le voie. `AuditVocabulary` est le mécanisme qui rend l'exigence réelle. +2. **`PinnedClock` reste privé, dupliqué dans le CLI et dans les Agents** (4 lignes chacun), comme le demande la spec §5. Un `PinnedClock` public dans le cœur supprimerait les deux copies (et la troisième, `FixedClock`, côté tests) mais élargirait la surface publique hors périmètre. Noté comme amélioration séparée possible, non faite ici. + +## Structure des fichiers + +| Fichier | Rôle | Task | +|---|---|---| +| `src/OKF4net.Cli/OkfCli.cs` (modifié) | `FlagValue` + `Positional(…, valuedFlags)` — prérequis de parsing | 0 | +| `src/OKF4net/Audit.cs` (créé) | `AuditQuery`, `AuditFinding`, `AuditReport`, `ConceptAudit`, `AuditVocabulary` — le calcul et le vocabulaire partagés | 1 | +| `tests/OKF4net.Tests/AuditTests.cs` (créé) | Tests unitaires du cœur | 1 | +| `src/OKF4net.Cli/OkfCli.cs` (modifié) | Usage, dispatch, `CmdAudit`, parsing des flags, rendu texte | 2 | +| `tests/OKF4net.Tests/CliTests.cs` (modifié) | Tests CLI texte + erreurs + parsing | 2 | +| `src/OKF4net.Cli/JsonOutput.cs` (modifié) | Records JSON, `WriteAudit`, enregistrement dans `CliJsonContext` | 3 | +| `tests/fixtures/golden/audit-v02.out` (créé) | Golden texte | 4 | +| `tests/fixtures/golden/audit-v02.json` (créé) | Golden JSON | 4 | +| `tests/OKF4net.Tests/GoldenParityTests.cs` (modifié) | Comparaison byte-exact + provenance | 4 | +| `tests/fixtures/README.md` (modifié) | Provenance des deux goldens | 4 | +| `src/OKF4net.Agents/OkfBundleTools.cs` (modifié) | Tool `okf_audit` + adaptateur d'horloge | 5 | +| `tests/OKF4net.Tests/Agents/OkfAuditToolTests.cs` (créé) | Tests du tool | 5 | +| `README.md`, `CHANGELOG.md` (modifiés) | Documentation | 6 | + +`ROADMAP.md` et `CLAUDE.md` sont **délibérément exclus** : ils sont modifiés en parallèle par une autre session (spec §8, « Coordination »). Ils seront traités en fin de branche, après merge de l'autre travail. + +--- + +### Task 0: Les helpers de parsing prérequis + +**Files:** +- Modify: `src/OKF4net.Cli/OkfCli.cs` +- Test: `tests/OKF4net.Tests/CliTests.cs` + +**Interfaces:** +- Consumes: rien. +- Produces: `FlagValue(string[] args, string flag) → string?` (throw `CliOperationException` si le flag est présent sans valeur) et `Positional(string[] args, string what, params string[] valuedFlags)`. Les Tasks 2 et 3 en dépendent. + +**Code identique à la branche viewer — ne pas le réécrire autrement.** Si cette +branche est un jour rebasée sur une base qui contient déjà ces helpers, supprimer +purement et simplement cette task : elle devient un no-op. + +- [ ] **Step 1: Écrire les tests qui échouent** + +Ajouter à `tests/OKF4net.Tests/CliTests.cs` : + +```csharp + /// + /// A valued flag's value must never be mistaken for the positional + /// argument. Without the valuedFlags declaration, `--as-of` placed + /// before the bundle path swallows the path's slot. + /// + [Fact] + public void Valued_flag_value_is_not_taken_for_the_positional() + { + // `fmt` takes a file positional; the flag here is unknown to it, which + // is fine: the point is which token Positional returns. + var r = Run("fmt", "--as-of", "2099-06-01", Path.Combine(BundlePath, "tables", "users.md")); + + Assert.Equal(0, r.Code); + } + + [Fact] + public void Missing_positional_reports_the_expected_error() + { + var r = Run("validate"); + + Assert.Equal(1, r.Code); + Assert.Equal("error: missing \n", r.Err); + } +``` + +Le premier test échouera tant que `fmt` ne déclare pas `--as-of` en `valuedFlags` +— ce n'est pas son rôle. **Utiliser plutôt ce test une fois `audit` livré** +(Task 2 le couvre déjà avec la `[Theory]` sur les quatre flags valués). Pour la +Task 0, se limiter au second test plus à une vérification de compilation : les +helpers sont du code d'infrastructure dont les Tasks 2 et 3 sont les vrais +consommateurs, et leur couverture réelle arrive là. + +- [ ] **Step 2: Lancer le test pour vérifier l'état de départ** + +Run: `dotnet test OKF4net.sln --filter "FullyQualifiedName~CliTests.Missing_positional"` +Expected: PASS (comportement déjà présent) — il sert de garde-fou pendant la modification de `Positional`. + +- [ ] **Step 3: Ajouter les helpers** + +Dans `src/OKF4net.Cli/OkfCli.cs`, remplacer la signature et le corps de `Positional` : + +```csharp + /// + /// Returns the first positional argument, or throws. Everything after a + /// -- separator is treated as positional (so paths beginning with + /// - work). + /// + /// The command's argument list. + /// Description of the missing positional, used in the error message. + /// + /// Flags that consume the following token as their value (e.g. --as-of) + /// rather than as a candidate positional. Verbs whose flags are all valueless + /// pass nothing, and the scan behaves exactly as before. + /// + private static string Positional(string[] args, string what, params string[] valuedFlags) + { + var sepIdx = Array.IndexOf(args, "--"); + if (sepIdx >= 0 && sepIdx + 1 < args.Length) + { + return args[sepIdx + 1]; + } + + for (var i = 0; i < args.Length; i++) + { + var a = args[i]; + if (Array.IndexOf(valuedFlags, a) >= 0) + { + i++; // Skip the value that belongs to this flag. + continue; + } + + if (!a.StartsWith('-')) + { + return a; + } + } + + throw new CliOperationException($"missing {what}"); + } +``` + +et ajouter, juste après `HasFlag` : + +```csharp + /// + /// The value following , or null when the + /// flag is absent. Throws when the flag is present but unvalued. + /// + private static string? FlagValue(string[] args, string flag) + { + var index = Array.IndexOf(args, flag); + if (index < 0) + { + return null; + } + + if (index + 1 >= args.Length) + { + throw new CliOperationException($"{flag} requires a value"); + } + + return args[index + 1]; + } +``` + +`FlagValue` n'a encore aucun appelant à ce stade. C'est volontaire : la Task 2 +est son consommateur. Le compilateur ne s'en plaint pas (une méthode privée +inutilisée ne produit pas de warning dans ce projet ; si un analyseur devait le +signaler, fusionner cette task avec la Task 2 plutôt que d'ajouter une +suppression). + +- [ ] **Step 4: Vérifier la non-régression** + +Run: `dotnet test OKF4net.sln` +Expected: 912 tests, 0 échec — aucun verbe existant ne change de comportement, +`valuedFlags` étant vide pour tous. + +- [ ] **Step 5: Commit** + +```bash +git add src/OKF4net.Cli/OkfCli.cs tests/OKF4net.Tests/CliTests.cs +git commit -m "refactor(cli): add FlagValue and valued-flag-aware Positional" +``` + +--- + +### Task 1: Le calcul partagé (`ConceptAudit`) + +**Files:** +- Create: `src/OKF4net/Audit.cs` +- Test: `tests/OKF4net.Tests/AuditTests.cs` + +**Interfaces:** +- Consumes: `Bundle.Concepts` (`IReadOnlyList`, chaque `Concept` porte `Id`, `Path`, `Document`), `Frontmatter.TrustTier`, `Frontmatter.Lifecycle`, `Frontmatter.Type`, `Frontmatter.Title`, `Lifecycle.IsStale(DateOnly)`, `Lifecycle.StaleAfter`, `Lifecycle.StaleAfterRaw`, `Lifecycle.Status`, `IOkfClock`/`SystemClock`. +- Produces: `ConceptAudit.Run(Bundle, AuditQuery, IOkfClock?) → AuditReport` ; `AuditQuery(bool StaleOnly, IReadOnlySet? Trust, ConceptStatus? Status, string? Type)` avec `AuditQuery.All` et `IsFiltered` ; `AuditFinding(ConceptId Id, string Path, string? Type, string? Title, TrustTier Trust, Lifecycle Lifecycle, bool IsStale)` ; `AuditReport` avec `AsOf`, `ConceptCount`, `TrustCounts`, `StatusCounts`, `StaleCount`, `Findings` ; `AuditVocabulary.Name(TrustTier)`, `Name(ConceptStatus)`, `TryParseTrustTier`, `TryParseStatus`, `TrustTierNames`, `StatusNames`. + +- [ ] **Step 1: Écrire les tests unitaires qui échouent** + +Créer `tests/OKF4net.Tests/AuditTests.cs` : + +```csharp +// SPDX-License-Identifier: LGPL-3.0-or-later +namespace OKF4net.Tests; + +/// +/// Unit tests for : tier and status counting, +/// the §5.5 staleness boundary, predicate composition and ordering. +/// Every test pins the date with so nothing here +/// depends on the day the suite runs. +/// +public class AuditTests +{ + private static readonly DateOnly Today = new(2026, 8, 21); + + private static Bundle Load(TempDir tmp) => Bundle.Load(tmp.Path); + + private static AuditReport Audit(TempDir tmp, AuditQuery query = default) + => ConceptAudit.Run(Load(tmp), query, new FixedClock(Today)); + + [Fact] + public void Counts_the_three_trust_tiers() + { + using var tmp = new TempDir(); + tmp.Write("a.md", "---\ntype: Metric\nverified:\n - { by: human:ada, at: 2026-01-01T00:00:00Z }\n---\n"); + tmp.Write("b.md", "---\ntype: Metric\nverified:\n - { by: process:nightly, at: 2026-01-01T00:00:00Z }\n---\n"); + tmp.Write("c.md", "---\ntype: Metric\n---\n"); + + var report = Audit(tmp); + + Assert.Equal(3, report.ConceptCount); + Assert.Equal(1, report.TrustCounts[TrustTier.HumanReviewed]); + Assert.Equal(1, report.TrustCounts[TrustTier.MachineConfirmed]); + Assert.Equal(1, report.TrustCounts[TrustTier.Unverified]); + } + + [Fact] + public void Unknown_status_counts_as_stable() + { + using var tmp = new TempDir(); + tmp.Write("a.md", "---\ntype: Metric\nstatus: retired\n---\n"); + tmp.Write("b.md", "---\ntype: Metric\nstatus: draft\n---\n"); + + var report = Audit(tmp); + + Assert.Equal(1, report.StatusCounts[ConceptStatus.Stable]); + Assert.Equal(1, report.StatusCounts[ConceptStatus.Draft]); + Assert.Equal(0, report.StatusCounts[ConceptStatus.Deprecated]); + } + + [Theory] + [InlineData("2026-08-21", true)] // §5.5: today >= stale_after -- the exact boundary IS stale. + [InlineData("2026-08-22", false)] + public void Staleness_boundary_follows_section_5_5(string staleAfter, bool expectedStale) + { + using var tmp = new TempDir(); + tmp.Write("a.md", $"---\ntype: Metric\nstale_after: {staleAfter}\n---\n"); + + // The default query filters nothing, so the single concept is always returned. + var report = Audit(tmp); + + Assert.Equal(expectedStale, report.Findings.Single().IsStale); + Assert.Equal(expectedStale ? 1 : 0, report.StaleCount); + } + + [Fact] + public void Malformed_or_absent_stale_after_is_never_stale() + { + using var tmp = new TempDir(); + tmp.Write("bad.md", "---\ntype: Metric\nstale_after: not-a-date\n---\n"); + tmp.Write("none.md", "---\ntype: Metric\n---\n"); + + var report = Audit(tmp); + + Assert.Equal(0, report.StaleCount); + Assert.Empty(ConceptAudit.Run(Load(tmp), new AuditQuery(StaleOnly: true), new FixedClock(Today)).Findings); + Assert.True(report.Findings.Single(f => f.Id.ToString() == "bad").Lifecycle.StaleAfterMalformed); + } + + [Fact] + public void Predicates_compose_with_and() + { + using var tmp = new TempDir(); + tmp.Write("stale-unverified.md", "---\ntype: Metric\nstale_after: 2026-01-01\n---\n"); + tmp.Write("stale-human.md", "---\ntype: Metric\nstale_after: 2026-01-01\nverified:\n - { by: human:ada, at: 2026-01-01T00:00:00Z }\n---\n"); + tmp.Write("fresh-unverified.md", "---\ntype: Metric\nstale_after: 2099-01-01\n---\n"); + + var query = new AuditQuery( + StaleOnly: true, + Trust: new HashSet { TrustTier.Unverified }); + + var findings = ConceptAudit.Run(Load(tmp), query, new FixedClock(Today)).Findings; + + Assert.Equal(["stale-unverified"], findings.Select(f => f.Id.ToString())); + } + + [Fact] + public void Findings_are_sorted_by_concept_id_ordinal() + { + using var tmp = new TempDir(); + tmp.Write("zeta.md", "---\ntype: Metric\n---\n"); + tmp.Write("alpha.md", "---\ntype: Metric\n---\n"); + tmp.Write("mid/beta.md", "---\ntype: Metric\n---\n"); + + var findings = Audit(tmp).Findings.Select(f => f.Id.ToString()).ToList(); + + Assert.Equal(["alpha", "mid/beta", "zeta"], findings); + } + + [Fact] + public void Counts_cover_the_whole_bundle_even_when_the_query_filters() + { + using var tmp = new TempDir(); + tmp.Write("stale.md", "---\ntype: Metric\nstale_after: 2026-01-01\n---\n"); + tmp.Write("fresh.md", "---\ntype: Metric\n---\n"); + + var report = ConceptAudit.Run(Load(tmp), new AuditQuery(StaleOnly: true), new FixedClock(Today)); + + Assert.Single(report.Findings); + Assert.Equal(2, report.ConceptCount); + Assert.Equal(2, report.TrustCounts[TrustTier.Unverified]); + Assert.Equal(1, report.StaleCount); + } + + [Fact] + public void Empty_bundle_yields_zeroed_counts_and_no_findings() + { + using var tmp = new TempDir(); + + var report = Audit(tmp); + + Assert.Equal(0, report.ConceptCount); + Assert.Empty(report.Findings); + Assert.Equal(0, report.TrustCounts[TrustTier.HumanReviewed]); + Assert.Equal(0, report.StatusCounts[ConceptStatus.Deprecated]); + } + + /// + /// A document whose frontmatter cannot be parsed lands in + /// Bundle.ParseErrors (permissive loading) and is not a concept, so + /// it must not reach any counter. Note this is a *parse* failure -- a truly + /// unreadable file (I/O, permissions, non-UTF-8) throws + /// BundleLoadException and never reaches . + /// + [Fact] + public void Unparseable_documents_are_excluded_from_every_count() + { + using var tmp = new TempDir(); + tmp.Write("ok.md", "---\ntype: Metric\n---\n"); + tmp.Write("broken.md", "---\ntype: Metric\n"); // unterminated frontmatter block + + var bundle = Load(tmp); + Assert.Single(bundle.ParseErrors); + + var report = ConceptAudit.Run(bundle, default, new FixedClock(Today)); + + Assert.Equal(1, report.ConceptCount); + Assert.Single(report.Findings); + } + + [Fact] + public void Null_clock_falls_back_to_today_in_utc() + { + using var tmp = new TempDir(); + tmp.Write("a.md", "---\ntype: Metric\n---\n"); + + var report = ConceptAudit.Run(Load(tmp)); + + Assert.Equal(DateOnly.FromDateTime(DateTime.UtcNow.Date), report.AsOf); + } + + [Fact] + public void Type_filter_is_exact_and_ordinal() + { + using var tmp = new TempDir(); + tmp.Write("metric.md", "---\ntype: Metric\n---\n"); + tmp.Write("lower.md", "---\ntype: metric\n---\n"); + tmp.Write("untyped.md", "---\ntitle: No type here\n---\n"); + + var findings = ConceptAudit.Run(Load(tmp), new AuditQuery(Type: "Metric"), new FixedClock(Today)).Findings; + + Assert.Equal(["metric"], findings.Select(f => f.Id.ToString())); + } + + [Fact] + public void Vocabulary_names_round_trip() + { + Assert.Equal("human-reviewed", AuditVocabulary.Name(TrustTier.HumanReviewed)); + Assert.Equal("machine-confirmed", AuditVocabulary.Name(TrustTier.MachineConfirmed)); + Assert.Equal("unverified", AuditVocabulary.Name(TrustTier.Unverified)); + Assert.Equal("draft", AuditVocabulary.Name(ConceptStatus.Draft)); + + Assert.True(AuditVocabulary.TryParseTrustTier("machine-confirmed", out var tier)); + Assert.Equal(TrustTier.MachineConfirmed, tier); + Assert.False(AuditVocabulary.TryParseTrustTier("machine", out _)); + + Assert.True(AuditVocabulary.TryParseStatus("deprecated", out var status)); + Assert.Equal(ConceptStatus.Deprecated, status); + Assert.False(AuditVocabulary.TryParseStatus("retired", out _)); + } +} +``` + +- [ ] **Step 2: Lancer les tests pour vérifier qu'ils échouent** + +Run: `dotnet test OKF4net.sln --filter "FullyQualifiedName~AuditTests"` +Expected: échec de compilation — `ConceptAudit`, `AuditQuery`, `AuditReport`, `AuditVocabulary` n'existent pas. + +- [ ] **Step 3: Implémenter `src/OKF4net/Audit.cs`** + +```csharp +// SPDX-License-Identifier: LGPL-3.0-or-later +namespace OKF4net; + +/// +/// The selection predicates of an audit (§5.3–§5.5). Predicates combine with +/// AND; default selects every concept. +/// +/// +/// The generated equality compares by reference (the +/// behaviour of EqualityComparer<IReadOnlySet<T>>.Default), so two +/// logically identical queries may compare unequal. Do not rely on it, and do +/// not use an as a dictionary key: the record struct is +/// for with and ToString, not for its equality. +/// +/// Keep only concepts past their stale_after date. +/// Keep only concepts whose derived tier is in this set; null keeps every tier. +/// Keep only concepts with this lifecycle status; null keeps every status. +/// Keep only concepts whose frontmatter type matches exactly (ordinal); null keeps every type. +public readonly record struct AuditQuery( + bool StaleOnly = false, + IReadOnlySet? Trust = null, + ConceptStatus? Status = null, + string? Type = null) +{ + /// The query that keeps every concept. + public static AuditQuery All => default; + + /// True as soon as one predicate is set. + public bool IsFiltered => StaleOnly || Trust is not null || Status is not null || Type is not null; +} + +/// One concept selected by an audit, with its signals already derived. +/// The concept id (§2). +/// The concept's file path, as built from the bundle root. +/// The frontmatter type, or null when absent. +/// The frontmatter title, or null when absent. +/// The derived trust tier (§5.3). +/// The lifecycle fields (§5.4/§5.5). +/// Whether the concept is stale as of the report's . +public readonly record struct AuditFinding( + ConceptId Id, + string Path, + string? Type, + string? Title, + TrustTier Trust, + Lifecycle Lifecycle, + bool IsStale); + +/// +/// The result of an audit: counts over the whole bundle, plus the concepts the +/// query selected. The counts never narrow with the query -- the denominator +/// stays stable while moves. +/// +public sealed class AuditReport +{ + internal AuditReport( + DateOnly asOf, + int conceptCount, + IReadOnlyDictionary trustCounts, + IReadOnlyDictionary statusCounts, + int staleCount, + IReadOnlyList findings) + { + AsOf = asOf; + ConceptCount = conceptCount; + TrustCounts = trustCounts; + StatusCounts = statusCounts; + StaleCount = staleCount; + Findings = findings; + } + + /// The observation date staleness was evaluated against. + public DateOnly AsOf { get; } + + /// The number of concepts in the bundle (not in the selection). + public int ConceptCount { get; } + + /// Concept counts per trust tier over the whole bundle; all three keys are always present. + public IReadOnlyDictionary TrustCounts { get; } + + /// Concept counts per lifecycle status over the whole bundle; all three keys are always present. + public IReadOnlyDictionary StatusCounts { get; } + + /// The number of stale concepts in the whole bundle. + public int StaleCount { get; } + + /// The selected concepts, sorted by concept id (ordinal). + public IReadOnlyList Findings { get; } +} + +/// +/// The single spelling of the audit vocabularies, shared by every surface (CLI +/// input, CLI text, JSON, agent tool) so no two layers can drift apart. +/// +public static class AuditVocabulary +{ + /// The trust tiers, weakest to strongest -- the canonical serialization order. + public static IReadOnlyList TrustTiersInOrder { get; } = + [TrustTier.Unverified, TrustTier.MachineConfirmed, TrustTier.HumanReviewed]; + + /// The wire/display name of a trust tier. + public static string Name(TrustTier tier) => tier switch + { + TrustTier.HumanReviewed => "human-reviewed", + TrustTier.MachineConfirmed => "machine-confirmed", + _ => "unverified", + }; + + /// The wire/display name of a lifecycle status. + public static string Name(ConceptStatus status) => status switch + { + ConceptStatus.Draft => "draft", + ConceptStatus.Deprecated => "deprecated", + _ => "stable", + }; + + /// Parses a trust tier name (exact, ordinal). Unlike frontmatter parsing, an unknown name fails rather than defaulting. + public static bool TryParseTrustTier(string text, out TrustTier tier) + { + switch (text) + { + case "unverified": tier = TrustTier.Unverified; return true; + case "machine-confirmed": tier = TrustTier.MachineConfirmed; return true; + case "human-reviewed": tier = TrustTier.HumanReviewed; return true; + default: tier = TrustTier.Unverified; return false; + } + } + + /// Parses a lifecycle status name (exact, ordinal). Unlike , an unknown name fails rather than resolving to stable. + public static bool TryParseStatus(string text, out ConceptStatus status) + { + switch (text) + { + case "draft": status = ConceptStatus.Draft; return true; + case "stable": status = ConceptStatus.Stable; return true; + case "deprecated": status = ConceptStatus.Deprecated; return true; + default: status = ConceptStatus.Stable; return false; + } + } +} + +/// +/// Queries a bundle's §5.3–§5.5 signals. Reads nothing from disk, writes +/// nothing, and never throws on data: an unparseable document is already absent +/// from , and a malformed stale_after simply +/// reads as "not stale" (the validator owns that diagnostic). +/// +public static class ConceptAudit +{ + /// Runs over . + /// The loaded bundle. + /// The selection predicates; default selects everything. + /// Supplies "today" for staleness (§5.5); defaults to . + public static AuditReport Run(Bundle bundle, AuditQuery query = default, IOkfClock? clock = null) + { + var asOf = (clock ?? new SystemClock()).Today; + + var trustCounts = new Dictionary + { + [TrustTier.Unverified] = 0, + [TrustTier.MachineConfirmed] = 0, + [TrustTier.HumanReviewed] = 0, + }; + + var statusCounts = new Dictionary + { + [ConceptStatus.Draft] = 0, + [ConceptStatus.Stable] = 0, + [ConceptStatus.Deprecated] = 0, + }; + + var staleCount = 0; + var findings = new List(); + + foreach (var concept in bundle.Concepts) + { + var frontmatter = concept.Document.Frontmatter; + var tier = frontmatter.TrustTier; + var lifecycle = frontmatter.Lifecycle; + var isStale = lifecycle.IsStale(asOf); + + trustCounts[tier]++; + statusCounts[lifecycle.Status]++; + if (isStale) + { + staleCount++; + } + + if (query.StaleOnly && !isStale) + { + continue; + } + + if (query.Trust is { } tiers && !tiers.Contains(tier)) + { + continue; + } + + if (query.Status is { } status && lifecycle.Status != status) + { + continue; + } + + if (query.Type is { } type && !string.Equals(frontmatter.Type, type, StringComparison.Ordinal)) + { + continue; + } + + findings.Add(new AuditFinding( + concept.Id, + concept.Path, + frontmatter.Type, + frontmatter.Title, + tier, + lifecycle, + isStale)); + } + + findings.Sort(static (a, b) => string.CompareOrdinal(a.Id.ToString(), b.Id.ToString())); + + return new AuditReport(asOf, bundle.Count, trustCounts, statusCounts, staleCount, findings); + } +} +``` + +- [ ] **Step 4: Lancer les tests pour vérifier qu'ils passent** + +Run: `dotnet test OKF4net.sln --filter "FullyQualifiedName~AuditTests"` +Expected: PASS — 12 méthodes de test, 13 cas exécutés (la `[Theory]` en compte +deux). Puis `dotnet test OKF4net.sln` en entier : 912 tests de base + les +nouveaux, 0 échec. + +- [ ] **Step 5: Commit** + +```bash +git add src/OKF4net/Audit.cs tests/OKF4net.Tests/AuditTests.cs +git commit -m "feat(audit): add ConceptAudit, the shared corpus-level query" +``` + +--- + +### Task 2: Le verbe CLI `okf audit` (sortie texte) + +**Files:** +- Modify: `src/OKF4net.Cli/OkfCli.cs` (constante `Usage`, `switch` de `Run`, nouvelles méthodes privées) +- Test: `tests/OKF4net.Tests/CliTests.cs` + +**Interfaces:** +- Consumes: Task 1 (`ConceptAudit.Run`, `AuditQuery`, `AuditReport`, `AuditFinding`, `AuditVocabulary`), plus les helpers existants `Positional(args, what, valuedFlags)`, `HasFlag`, `FlagValue`, `Load(path)`, `CliOperationException`. +- Produces: le verbe `audit` et son format texte, dont les Tasks 3 et 4 dépendent. + +- [ ] **Step 1: Écrire les tests qui échouent** + +Ajouter à `tests/OKF4net.Tests/CliTests.cs` (dans la classe `CliTests`) : + +```csharp + private static readonly string V02BundlePath = + Path.Combine(TestPaths.RepoRoot(), "tests", "fixtures", "okf_v02"); + + [Fact] + public void Audit_report_mode_prints_summary_and_worklist() + { + var r = Run("audit", V02BundlePath, "--as-of", "2099-06-01"); + + Assert.Equal(0, r.Code); + Assert.Contains("as of: 2099-06-01\n", r.Out); + Assert.Contains("concepts: 2\n", r.Out); + Assert.Contains(" 1 human-reviewed\n", r.Out); + Assert.Contains(" 1 unverified\n", r.Out); + Assert.Contains(" 2 stable\n", r.Out); + Assert.Contains("stale: 1 of 2 past stale_after\n", r.Out); + Assert.Contains("needs attention (1):\n", r.Out); + Assert.Contains(" metrics/dau stale 2099-01-01 human-reviewed stable\n", r.Out); + } + + [Fact] + public void Audit_query_mode_prints_bare_lines_only() + { + var r = Run("audit", V02BundlePath, "--stale", "--as-of", "2099-06-01"); + + Assert.Equal(0, r.Code); + Assert.Equal("metrics/dau stale 2099-01-01 human-reviewed stable\n", r.Out); + } + + [Fact] + public void Audit_without_flags_selects_the_same_set_as_stale() + { + var report = Run("audit", V02BundlePath, "--as-of", "2099-06-01"); + var query = Run("audit", V02BundlePath, "--stale", "--as-of", "2099-06-01"); + + var reportIds = report.Out + .Split('\n') + .Where(l => l.StartsWith(" metrics/", StringComparison.Ordinal)) + .Select(l => l.Trim().Split(" ")[0]) + .ToList(); + var queryIds = query.Out + .Split('\n', StringSplitOptions.RemoveEmptyEntries) + .Select(l => l.Split(" ")[0]) + .ToList(); + + Assert.Equal(queryIds, reportIds); + } + + [Fact] + public void Audit_empty_selection_prints_nothing() + { + var r = Run("audit", V02BundlePath, "--status", "deprecated"); + + Assert.Equal(0, r.Code); + Assert.Equal("", r.Out); + } + + [Fact] + public void Audit_three_tier_idiom_returns_every_concept() + { + var r = Run("audit", V02BundlePath, "--trust", "unverified,machine-confirmed,human-reviewed"); + + Assert.Equal(0, r.Code); + Assert.Equal(2, r.Out.Split('\n', StringSplitOptions.RemoveEmptyEntries).Length); + } + + [Fact] + public void Audit_rejects_an_invalid_as_of_date() + { + var r = Run("audit", V02BundlePath, "--as-of", "2026-13-01"); + + Assert.Equal(1, r.Code); + Assert.Equal("error: --as-of is not a valid YYYY-MM-DD date: \"2026-13-01\"\n", r.Err); + } + + [Fact] + public void Audit_rejects_an_unknown_trust_tier() + { + var r = Run("audit", V02BundlePath, "--trust", "foo"); + + Assert.Equal(1, r.Code); + Assert.Equal( + "error: unknown trust tier \"foo\"; expected unverified, machine-confirmed or human-reviewed\n", + r.Err); + } + + [Fact] + public void Audit_rejects_an_unknown_status() + { + var r = Run("audit", V02BundlePath, "--status", "retired"); + + Assert.Equal(1, r.Code); + Assert.Equal("error: unknown status \"retired\"; expected draft, stable or deprecated\n", r.Err); + } + + [Fact] + public void Audit_rejects_an_empty_trust_entry_but_absorbs_duplicates() + { + var empty = Run("audit", V02BundlePath, "--trust", "unverified,,human-reviewed"); + Assert.Equal(1, empty.Code); + Assert.Contains("unknown trust tier", empty.Err); + + var duplicated = Run("audit", V02BundlePath, "--trust", "unverified,unverified"); + var single = Run("audit", V02BundlePath, "--trust", "unverified"); + Assert.Equal(0, duplicated.Code); + Assert.Equal(single.Out, duplicated.Out); + } + + /// + /// Regression guard: valued flags must be declared to Positional, or + /// their value is mistaken for the bundle path when they precede it. + /// + [Theory] + [InlineData("--as-of", "2099-06-01")] + [InlineData("--trust", "unverified")] + [InlineData("--status", "stable")] + [InlineData("--type", "Metric")] + public void Audit_valued_flags_before_the_positional_resolve_the_bundle(string flag, string value) + { + var r = Run("audit", flag, value, V02BundlePath); + + Assert.Equal(0, r.Code); + Assert.Equal("", r.Err); + } + + [Fact] + public void Audit_as_of_alone_stays_in_report_mode() + { + var r = Run("audit", V02BundlePath, "--as-of", "2099-06-01"); + + Assert.Contains("needs attention", r.Out); + } + + [Fact] + public void Help_lists_audit_right_after_validate() + { + var r = Run("--help"); + + Assert.Equal(0, r.Code); + var lines = r.Out.Split('\n').Select(l => l.TrimStart()).ToList(); + var validateIndex = lines.FindIndex(l => l.StartsWith("validate ", StringComparison.Ordinal)); + var auditIndex = lines.FindIndex(l => l.StartsWith("audit ", StringComparison.Ordinal)); + + Assert.True(validateIndex >= 0 && auditIndex == validateIndex + 1); + } +``` + +- [ ] **Step 2: Lancer les tests pour vérifier qu'ils échouent** + +Run: `dotnet test OKF4net.sln --filter "FullyQualifiedName~CliTests"` +Expected: échecs — `unknown subcommand: audit` sur chaque test d'audit. + +- [ ] **Step 3: Implémenter le verbe** + +Dans `src/OKF4net.Cli/OkfCli.cs`, ajouter `using System.Globalization;` en tête, puis : + +a) Dans la constante `Usage`, insérer la ligne **juste après** celle de `validate` : + +```csharp + " audit Report trust, freshness and lifecycle across the bundle\n" + +``` + +et remplacer la ligne d'option `--json` par : + +```csharp + " --json Machine-readable output for validate/info/audit\n" + +``` + +Mettre aussi à jour le commentaire XML de la classe : sur cette base il annonce +« Six subcommands » — il passe à sept, en citant `audit`. (Sur une base où le +viewer est mergé, ce serait sept → huit : compter les verbes réellement présents +plutôt que recopier ce chiffre.) + +b) Dans le `switch` de `Run`, ajouter après `"validate"` : + +```csharp + "audit" => CmdAudit(rest, stdout), +``` + +c) Ajouter les membres privés : + +```csharp + /// The flags that make audit a filtered query rather than a report. + private static readonly string[] AuditFilterFlags = ["--stale", "--trust", "--status", "--type"]; + + /// Every audit flag that consumes the following token as its value. + private static readonly string[] AuditValuedFlags = ["--trust", "--status", "--type", "--as-of"]; + + /// An pinned to one date, backing --as-of. + private sealed class PinnedClock(DateOnly today) : IOkfClock + { + public DateOnly Today { get; } = today; + } + + /// Implements the audit subcommand. + private static int CmdAudit(string[] args, TextWriter stdout) + { + var path = Positional(args, "", AuditValuedFlags); + var clock = ParseAsOf(args); + + // Report mode selects exactly what --stale selects; only the + // presentation differs. --as-of and --json never switch modes. + var filtered = AuditFilterFlags.Any(f => HasFlag(args, f)); + var query = filtered ? ParseAuditQuery(args) : new AuditQuery(StaleOnly: true); + + var bundle = Load(path); + var report = ConceptAudit.Run(bundle, query, clock); + + if (HasFlag(args, "--json")) + { + JsonOutput.WriteAudit(stdout, path, query, report); + return 0; + } + + if (filtered) + { + foreach (var finding in report.Findings) + { + stdout.Write(FormatAuditFinding(finding)); + stdout.Write("\n"); + } + + return 0; + } + + WriteAuditReport(stdout, path, report); + return 0; + } + + /// Parses --as-of; null when absent (the audit then uses the system clock). + private static IOkfClock? ParseAsOf(string[] args) + { + var raw = FlagValue(args, "--as-of"); + if (raw is null) + { + return null; + } + + // DateOnly has no (s, format, provider, out) overload -- the five-argument + // form is the only one that takes a culture, and it is the same contract + // Lifecycle.From uses for stale_after. + if (!DateOnly.TryParseExact(raw, "yyyy-MM-dd", CultureInfo.InvariantCulture, DateTimeStyles.None, out var asOf)) + { + throw new CliOperationException($"--as-of is not a valid YYYY-MM-DD date: \"{raw}\""); + } + + return new PinnedClock(asOf); + } + + /// Builds the query from the filter flags. Throws on an unknown vocabulary value. + private static AuditQuery ParseAuditQuery(string[] args) + { + HashSet? tiers = null; + var trustRaw = FlagValue(args, "--trust"); + if (trustRaw is not null) + { + tiers = []; + foreach (var entry in trustRaw.Split(',')) + { + if (!AuditVocabulary.TryParseTrustTier(entry.Trim(), out var tier)) + { + throw new CliOperationException( + $"unknown trust tier \"{entry.Trim()}\"; expected unverified, machine-confirmed or human-reviewed"); + } + + tiers.Add(tier); + } + } + + ConceptStatus? status = null; + var statusRaw = FlagValue(args, "--status"); + if (statusRaw is not null) + { + if (!AuditVocabulary.TryParseStatus(statusRaw.Trim(), out var parsed)) + { + throw new CliOperationException( + $"unknown status \"{statusRaw.Trim()}\"; expected draft, stable or deprecated"); + } + + status = parsed; + } + + return new AuditQuery( + HasFlag(args, "--stale"), + tiers, + status, + FlagValue(args, "--type")); + } + + /// Renders one concept line: id, freshness, trust tier, status -- two spaces between fields. + private static string FormatAuditFinding(AuditFinding finding) + { + var freshness = finding.Lifecycle.StaleAfter is { } date + ? (finding.IsStale ? "stale " : "fresh ") + date.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture) + : "no-stale-after"; + + return $"{finding.Id} {freshness} {AuditVocabulary.Name(finding.Trust)} {AuditVocabulary.Name(finding.Lifecycle.Status)}"; + } + + /// Renders the report form: summary counters over the whole bundle, then the worklist. + private static void WriteAuditReport(TextWriter stdout, string bundlePath, AuditReport report) + { + stdout.Write($"bundle: {bundlePath}\n"); + stdout.Write($"as of: {report.AsOf.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture)}\n"); + stdout.Write($"concepts: {report.ConceptCount}\n"); + + stdout.Write("\ntrust:\n"); + stdout.Write($" {report.TrustCounts[TrustTier.HumanReviewed],4} human-reviewed\n"); + stdout.Write($" {report.TrustCounts[TrustTier.MachineConfirmed],4} machine-confirmed\n"); + stdout.Write($" {report.TrustCounts[TrustTier.Unverified],4} unverified\n"); + + stdout.Write("\nstatus:\n"); + stdout.Write($" {report.StatusCounts[ConceptStatus.Draft],4} draft\n"); + stdout.Write($" {report.StatusCounts[ConceptStatus.Stable],4} stable\n"); + stdout.Write($" {report.StatusCounts[ConceptStatus.Deprecated],4} deprecated\n"); + + stdout.Write($"\nstale: {report.StaleCount} of {report.ConceptCount} past stale_after\n"); + + if (report.Findings.Count == 0) + { + stdout.Write("\nneeds attention: none\n"); + return; + } + + stdout.Write($"\nneeds attention ({report.Findings.Count}):\n"); + foreach (var finding in report.Findings) + { + stdout.Write(" "); + stdout.Write(FormatAuditFinding(finding)); + stdout.Write("\n"); + } + } +``` + +**Note pour la Task 3 :** `JsonOutput.WriteAudit` n'existe pas encore. Pour que la Task 2 compile seule, remplacer temporairement la branche `--json` par `throw new CliOperationException("--json is not implemented yet");` **et** ne pas écrire de test `--json` ici — la Task 3 rétablit l'appel réel. Si les deux tasks sont exécutées d'affilée, écrire directement l'appel final et implémenter la Task 3 avant de lancer les tests. + +- [ ] **Step 4: Lancer les tests pour vérifier qu'ils passent** + +Run: `dotnet test OKF4net.sln --filter "FullyQualifiedName~CliTests"` +Expected: PASS. Puis `dotnet test OKF4net.sln` en entier : aucun golden existant ne doit bouger. + +- [ ] **Step 5: Commit** + +```bash +git add src/OKF4net.Cli/OkfCli.cs tests/OKF4net.Tests/CliTests.cs +git commit -m "feat(cli): add the okf audit verb with its report and query modes" +``` + +--- + +### Task 3: La sortie `okf audit --json` + +**Files:** +- Modify: `src/OKF4net.Cli/JsonOutput.cs` +- Modify: `src/OKF4net.Cli/OkfCli.cs` (rétablir l'appel réel si la Task 2 a posé le stub) +- Test: `tests/OKF4net.Tests/CliTests.cs` + +**Interfaces:** +- Consumes: Task 1 (`AuditReport`, `AuditQuery`, `AuditVocabulary`), Task 2 (`CmdAudit`). +- Produces: `JsonOutput.WriteAudit(TextWriter stdout, string bundlePath, AuditQuery query, AuditReport report)` et le schéma JSON que la Task 4 fige en golden. + +- [ ] **Step 1: Écrire le test qui échoue** + +Ajouter à `tests/OKF4net.Tests/CliTests.cs` (ajouter `using System.Text.Json;` en tête du fichier si absent) : + +```csharp + [Fact] + public void Audit_json_carries_counts_query_and_findings() + { + var r = Run("audit", V02BundlePath, "--as-of", "2099-06-01", "--json"); + + Assert.Equal(0, r.Code); + Assert.EndsWith("\n", r.Out); + + using var doc = JsonDocument.Parse(r.Out); + var root = doc.RootElement; + + Assert.Equal("2099-06-01", root.GetProperty("asOf").GetString()); + Assert.Equal(2, root.GetProperty("conceptCount").GetInt32()); + Assert.Equal(1, root.GetProperty("staleCount").GetInt32()); + + // Report mode selects what --stale selects, so the replayed query says so. + Assert.True(root.GetProperty("query").GetProperty("stale").GetBoolean()); + Assert.Equal(JsonValueKind.Null, root.GetProperty("query").GetProperty("trust").ValueKind); + + Assert.Equal(1, root.GetProperty("trust").GetProperty("humanReviewed").GetInt32()); + Assert.Equal(1, root.GetProperty("trust").GetProperty("unverified").GetInt32()); + Assert.Equal(2, root.GetProperty("status").GetProperty("stable").GetInt32()); + + var finding = root.GetProperty("findings").EnumerateArray().Single(); + Assert.Equal("metrics/dau", finding.GetProperty("conceptId").GetString()); + Assert.Equal("Metric", finding.GetProperty("type").GetString()); + Assert.Equal("Daily Active Users", finding.GetProperty("title").GetString()); + Assert.Equal("human-reviewed", finding.GetProperty("trust").GetString()); + Assert.Equal("2099-01-01", finding.GetProperty("staleAfter").GetString()); + Assert.True(finding.GetProperty("stale").GetBoolean()); + } + + [Fact] + public void Audit_json_serializes_trust_query_in_ladder_order() + { + var r = Run("audit", V02BundlePath, "--trust", "human-reviewed,unverified", "--json"); + + using var doc = JsonDocument.Parse(r.Out); + var trust = doc.RootElement.GetProperty("query").GetProperty("trust") + .EnumerateArray().Select(e => e.GetString()).ToList(); + + Assert.Equal(["unverified", "human-reviewed"], trust); + } + + [Fact] + public void Audit_json_keeps_a_malformed_stale_after_raw_and_not_stale() + { + using var tmp = new TempDir(); + tmp.Write("a.md", "---\ntype: Metric\nstale_after: not-a-date\n---\n"); + + var r = Run("audit", tmp.Path, "--trust", "unverified", "--json"); + + using var doc = JsonDocument.Parse(r.Out); + var finding = doc.RootElement.GetProperty("findings").EnumerateArray().Single(); + + Assert.Equal("not-a-date", finding.GetProperty("staleAfter").GetString()); + Assert.False(finding.GetProperty("stale").GetBoolean()); + } +``` + +- [ ] **Step 2: Lancer les tests pour vérifier qu'ils échouent** + +Run: `dotnet test OKF4net.sln --filter "FullyQualifiedName~CliTests.Audit_json"` +Expected: échec (stub `--json is not implemented yet`, ou erreur de compilation sur `WriteAudit`). + +- [ ] **Step 3: Implémenter la sérialisation** + +Dans `src/OKF4net.Cli/JsonOutput.cs`, ajouter `using System.Globalization;` en tête, puis les records après `InfoJsonResult` : + +```csharp +/// The query okf audit applied, replayed for --json consumers. +internal sealed record AuditQueryJson(bool Stale, IReadOnlyList? Trust, string? Status, string? Type); + +/// Concept counts per trust tier (§5.3), over the whole bundle. +internal sealed record TrustCountsJson(int HumanReviewed, int MachineConfirmed, int Unverified); + +/// Concept counts per lifecycle status (§5.4), over the whole bundle. +internal sealed record StatusCountsJson(int Draft, int Stable, int Deprecated); + +/// One selected concept, projected for --json output. +internal sealed record AuditFindingJson( + string ConceptId, + string Path, + string? Type, + string? Title, + string Trust, + string Status, + string? StaleAfter, + bool Stale); + +/// The full result of okf audit --json. +internal sealed record AuditJsonResult( + string Bundle, + string AsOf, + int ConceptCount, + AuditQueryJson Query, + TrustCountsJson Trust, + StatusCountsJson Status, + int StaleCount, + IReadOnlyList Findings); +``` + +Ajouter l'attribut sur `CliJsonContext`, à côté des deux existants : + +```csharp +[JsonSerializable(typeof(AuditJsonResult))] +``` + +Puis la méthode, dans `JsonOutput` : + +```csharp + /// Writes okf audit --json's result to as a single line-terminated JSON document. + internal static void WriteAudit(TextWriter stdout, string bundlePath, AuditQuery query, AuditReport report) + { + // Serialized in ladder order, never in the order the user typed them: + // IReadOnlySet has no guaranteed order, and the document must be + // reproducible. Written as a statement rather than a ternary so the + // nullable analysis narrows `Trust` through the pattern -- it does not + // narrow a property across the arms of a conditional. + List? trustQuery = null; + if (query.Trust is { } selectedTiers) + { + trustQuery = AuditVocabulary.TrustTiersInOrder + .Where(selectedTiers.Contains) + .Select(AuditVocabulary.Name) + .ToList(); + } + + var findings = report.Findings + .Select(f => new AuditFindingJson( + f.Id.ToString(), + f.Path, + f.Type, + f.Title, + AuditVocabulary.Name(f.Trust), + AuditVocabulary.Name(f.Lifecycle.Status), + f.Lifecycle.StaleAfterRaw, + f.IsStale)) + .ToList(); + + var result = new AuditJsonResult( + bundlePath, + report.AsOf.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture), + report.ConceptCount, + new AuditQueryJson( + query.StaleOnly, + trustQuery, + query.Status is { } status ? AuditVocabulary.Name(status) : null, + query.Type), + new TrustCountsJson( + report.TrustCounts[TrustTier.HumanReviewed], + report.TrustCounts[TrustTier.MachineConfirmed], + report.TrustCounts[TrustTier.Unverified]), + new StatusCountsJson( + report.StatusCounts[ConceptStatus.Draft], + report.StatusCounts[ConceptStatus.Stable], + report.StatusCounts[ConceptStatus.Deprecated]), + report.StaleCount, + findings); + + stdout.Write(JsonSerializer.Serialize(result, CliJsonContext.Default.AuditJsonResult)); + stdout.Write("\n"); + } +``` + +Enfin, dans `OkfCli.CmdAudit`, remplacer le stub éventuel par l'appel réel : + +```csharp + if (HasFlag(args, "--json")) + { + JsonOutput.WriteAudit(stdout, path, query, report); + return 0; + } +``` + +- [ ] **Step 4: Lancer les tests pour vérifier qu'ils passent** + +Run: `dotnet test OKF4net.sln --filter "FullyQualifiedName~CliTests"` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add src/OKF4net.Cli/JsonOutput.cs src/OKF4net.Cli/OkfCli.cs tests/OKF4net.Tests/CliTests.cs +git commit -m "feat(cli): add machine-readable output to okf audit" +``` + +--- + +### Task 4: Les goldens byte-exact + +**Files:** +- Create: `tests/fixtures/golden/audit-v02.out` +- Create: `tests/fixtures/golden/audit-v02.json` +- Modify: `tests/OKF4net.Tests/GoldenParityTests.cs` +- Modify: `tests/fixtures/README.md` + +**Interfaces:** +- Consumes: Tasks 2 et 3 (les deux formats de sortie). +- Produces: rien pour les tasks suivantes — c'est un verrou de non-régression. + +**Rappel de règle :** aucun bundle de fixtures n'est créé ni modifié. `tests/fixtures/okf_v02` est réutilisé tel quel ; `--as-of 2099-06-01` suffit à rendre `metrics/dau` (`stale_after: 2099-01-01`) périmé. Ces deux goldens sont **écrits à la main** et vérifiés contre le texte du spec, **pas** capturés depuis un binaire de référence : `audit` n'existe pas en amont. + +- [ ] **Step 1: Écrire les deux goldens à la main** + +`tests/fixtures/golden/audit-v02.out` — **fins de ligne LF, pas de CRLF** (le `.gitattributes` marque `tests/fixtures/` en `-text`) : + +``` +bundle: tests/fixtures/okf_v02 +as of: 2099-06-01 +concepts: 2 + +trust: + 1 human-reviewed + 0 machine-confirmed + 1 unverified + +status: + 0 draft + 2 stable + 0 deprecated + +stale: 1 of 2 past stale_after + +needs attention (1): + metrics/dau stale 2099-01-01 human-reviewed stable +``` + +`tests/fixtures/golden/audit-v02.json` — une seule ligne, terminée par `\n` : + +``` +{"bundle":"tests/fixtures/okf_v02","asOf":"2099-06-01","conceptCount":2,"query":{"stale":true,"trust":null,"status":null,"type":null},"trust":{"humanReviewed":1,"machineConfirmed":0,"unverified":1},"status":{"draft":0,"stable":2,"deprecated":0},"staleCount":1,"findings":[{"conceptId":"metrics/dau","path":"tests/fixtures/okf_v02/metrics/dau.md","type":"Metric","title":"Daily Active Users","trust":"human-reviewed","status":"stable","staleAfter":"2099-01-01","stale":true}]} +``` + +- [ ] **Step 2: Écrire les tests de parité** + +Ajouter à `tests/OKF4net.Tests/GoldenParityTests.cs` : + +```csharp + /// + /// audit's goldens are hand-authored and verified against the spec + /// text (§5.3 tiers, §5.4 statuses, §5.5 staleness), not captured from the + /// reference CLI -- the verb has no upstream counterpart. The date is + /// pinned with --as-of so the output cannot drift with the calendar. + /// There is no audit-v02.exitcode: the verb always exits 0, so the + /// code is asserted inline (as does). + /// + [Fact] + public void Audit_report_matches_golden() + { + var r = WithRepoRootAsCwd(() => Run("audit", "tests/fixtures/okf_v02", "--as-of", "2099-06-01")); + + Assert.Equal(0, r.Code); + + // Every path in this output is a concept id, always '/'-normalized by + // ConceptId.FromPath, so the comparison is strict byte-for-byte. + Assert.Equal(Golden("audit-v02.out"), r.Out); + } + + [Fact] + public void Audit_json_matches_golden() + { + var r = WithRepoRootAsCwd(() => Run("audit", "tests/fixtures/okf_v02", "--as-of", "2099-06-01", "--json")); + + Assert.Equal(0, r.Code); + + // Only the findings' `path` field carries a native separator (it is a + // real file path, not a concept id), so it alone is normalized in the + // C# OUTPUT -- never in the golden -- exactly like validate.out. + Assert.Equal(Golden("audit-v02.json"), r.Out.Replace('\\', '/')); + } +``` + +- [ ] **Step 3: Lancer les tests de parité** + +Run: `dotnet test OKF4net.sln --filter "FullyQualifiedName~GoldenParityTests"` +Expected: PASS. **Si un écart apparaît, corriger le code du CLI, jamais le golden** — sauf si l'écart révèle une erreur dans ce plan, auquel cas corriger le golden ET le noter dans le message de commit. + +- [ ] **Step 4: Documenter la provenance** + +Dans `tests/fixtures/README.md`, ajouter à la liste des goldens : + +```markdown +- `golden/audit-v02.out`, `golden/audit-v02.json` — output of + `okf audit tests/fixtures/okf_v02 --as-of 2099-06-01` (and its `--json` + form). **Hand-authored**, verified against the spec text (§5.3 trust tiers, + §5.4 statuses, §5.5 staleness) rather than captured from the reference CLI: + `audit` is an OKF4net verb with no upstream counterpart. The `--as-of` date + is pinned so the output cannot drift with the calendar. +``` + +- [ ] **Step 5: Commit** + +```bash +git add tests/fixtures/golden/audit-v02.out tests/fixtures/golden/audit-v02.json \ + tests/OKF4net.Tests/GoldenParityTests.cs tests/fixtures/README.md +git commit -m "test(audit): pin okf audit's text and JSON output with goldens" +``` + +--- + +### Task 5: Le tool agent `okf_audit` + +**Files:** +- Modify: `src/OKF4net.Agents/OkfBundleTools.cs` +- Test: `tests/OKF4net.Tests/Agents/OkfAuditToolTests.cs` (créé) + +**Interfaces:** +- Consumes: Task 1 (`ConceptAudit.Run`, `AuditQuery`, `AuditReport`, `AuditVocabulary`), et les membres existants `GetBundle()`, `Today`, `UtcNow`, `GetTools()`, `WriteToolNames`. +- Produces: le tool `okf_audit` (lecture seule). + +- [ ] **Step 1: Écrire les tests qui échouent** + +Créer `tests/OKF4net.Tests/Agents/OkfAuditToolTests.cs` : + +```csharp +// SPDX-License-Identifier: LGPL-3.0-or-later +using Microsoft.Extensions.AI; +using OKF4net.Agents; + +namespace OKF4net.Tests.Agents; + +/// +/// Tests for the okf_audit tool. Every test pins UtcNow: the tool +/// deliberately exposes no asOf parameter, so the shared clock seam is +/// the only way its output can be made deterministic. +/// +public class OkfAuditToolTests +{ + private static OkfBundleTools ToolsOver(TempDir tmp, DateOnly today) + => new(tmp.Path) + { + UtcNow = () => new DateTime(today.Year, today.Month, today.Day, 0, 0, 0, DateTimeKind.Utc), + }; + + [Fact] + public void Audit_is_registered_and_read_only() + { + var tools = new OkfBundleTools(Path.Combine(TestPaths.RepoRoot(), "tests", "fixtures", "okf_v02")); + + Assert.Contains("okf_audit", tools.GetTools().OfType().Select(t => t.Name)); + Assert.DoesNotContain("okf_audit", OkfBundleTools.WriteToolNames); + } + + /// + /// §5.5's boundary is today >= stale_after, so a concept whose + /// stale_after is exactly today is stale. Without the pinned seam this + /// assertion would silently depend on the day the suite runs. + /// + [Fact] + public void Audit_treats_today_equals_stale_after_as_stale() + { + using var tmp = new TempDir(); + tmp.Write("a.md", "---\ntype: Metric\nstale_after: 2026-08-21\n---\n"); + + var onTheDay = ToolsOver(tmp, new DateOnly(2026, 8, 21)).Audit(); + var theDayBefore = ToolsOver(tmp, new DateOnly(2026, 8, 20)).Audit(); + + Assert.Contains("a stale 2026-08-21", onTheDay); + Assert.Contains("needs attention: none", theDayBefore); + } + + [Fact] + public void Audit_reports_counts_and_omits_the_bundle_line() + { + using var tmp = new TempDir(); + tmp.Write("a.md", "---\ntype: Metric\nverified:\n - { by: human:ada, at: 2026-01-01T00:00:00Z }\n---\n"); + tmp.Write("b.md", "---\ntype: Metric\n---\n"); + + var text = ToolsOver(tmp, new DateOnly(2026, 8, 21)).Audit(); + + Assert.DoesNotContain("bundle:", text); + Assert.Contains("as of: 2026-08-21", text); + Assert.Contains(" 1 human-reviewed", text); + Assert.Contains(" 1 unverified", text); + } + + [Fact] + public void Audit_caps_the_listing_at_twenty_findings() + { + using var tmp = new TempDir(); + for (var i = 0; i < 25; i++) + { + tmp.Write($"c{i:D2}.md", "---\ntype: Metric\nstale_after: 2026-01-01\n---\n"); + } + + var text = ToolsOver(tmp, new DateOnly(2026, 8, 21)).Audit(); + + Assert.Contains("… and 5 more (narrow with stale/trust/status/type)", text); + + // Finding lines are the two-space-indented ones; matching on "c" alone + // would also catch the "concepts:" header. + Assert.Equal(20, text.Split('\n').Count(l => l.StartsWith(" c", StringComparison.Ordinal))); + } + + [Fact] + public void Audit_renders_a_usage_message_for_invalid_vocabulary() + { + using var tmp = new TempDir(); + tmp.Write("a.md", "---\ntype: Metric\n---\n"); + + var tools = ToolsOver(tmp, new DateOnly(2026, 8, 21)); + + Assert.Contains("Usage: okf_audit", tools.Audit(trust: "machine")); + Assert.Contains("Usage: okf_audit", tools.Audit(status: "retired")); + } + + [Fact] + public void Audit_with_stale_false_and_no_filter_returns_the_whole_corpus() + { + using var tmp = new TempDir(); + tmp.Write("a.md", "---\ntype: Metric\n---\n"); + tmp.Write("b.md", "---\ntype: Metric\n---\n"); + + var text = ToolsOver(tmp, new DateOnly(2026, 8, 21)).Audit(stale: false); + + Assert.Contains("needs attention (2):", text); + } +} +``` + +- [ ] **Step 2: Lancer les tests pour vérifier qu'ils échouent** + +Run: `dotnet test OKF4net.sln --filter "FullyQualifiedName~OkfAuditToolTests"` +Expected: échec de compilation — `Audit` n'existe pas sur `OkfBundleTools`. + +- [ ] **Step 3: Implémenter le tool** + +Dans `src/OKF4net.Agents/OkfBundleTools.cs` : + +a) Ajouter la constante d'usage à côté de `SearchUsageMessage` : + +```csharp + private const string AuditUsageMessage = + "Usage: okf_audit takes optional filters — stale (bool), trust (comma-separated: " + + "unverified, machine-confirmed, human-reviewed), status (draft, stable or deprecated) " + + "and type (exact frontmatter type). Example: okf_audit(stale: true, trust: \"unverified\")."; +``` + +b) Ajouter l'adaptateur d'horloge, en membre privé de la classe : + +```csharp + /// + /// Pins to — the same + /// UtcNow seam and use — so + /// the tool's output never depends on the day it runs. + /// + private sealed class PinnedClock(DateOnly today) : IOkfClock + { + public DateOnly Today { get; } = today; + } +``` + +c) Ajouter la méthode du tool : + +```csharp + /// + /// Audits the bundle's trust, freshness and lifecycle signals (§5.3–§5.5): + /// counts over the whole bundle, then the concepts the filters select, + /// bounded to 20 entries. + /// + /// Keep only concepts past their stale_after date. + /// Comma-separated trust tiers to keep. + /// Keep only concepts with this lifecycle status. + /// Keep only concepts with this frontmatter type (exact match). + [Description("Audit the bundle's trust, freshness and lifecycle signals: counts by trust tier and status, plus the concepts needing attention. Filter with stale/trust/status/type.")] + public string Audit( + [Description("Only concepts past their stale_after date. Defaults to true.")] bool stale = true, + [Description("Comma-separated trust tiers to include: unverified, machine-confirmed, human-reviewed.")] string? trust = null, + [Description("Only concepts with this lifecycle status: draft, stable or deprecated.")] string? status = null, + [Description("Only concepts with this frontmatter type (exact match).")] string? type = null) + { + HashSet? tiers = null; + if (trust is not null) + { + tiers = []; + foreach (var entry in trust.Split(',')) + { + if (!AuditVocabulary.TryParseTrustTier(entry.Trim(), out var tier)) + { + return AuditUsageMessage; + } + + tiers.Add(tier); + } + } + + ConceptStatus? parsedStatus = null; + if (status is not null) + { + if (!AuditVocabulary.TryParseStatus(status.Trim(), out var value)) + { + return AuditUsageMessage; + } + + parsedStatus = value; + } + + var today = Today; + var report = ConceptAudit.Run( + GetBundle(), + new AuditQuery(stale, tiers, parsedStatus, type), + new PinnedClock(today)); + + return RenderAudit(report); + } + + /// + /// Renders an audit for an agent: the same shape as the CLI's report form, + /// minus the bundle line (the tool is bound to one bundle) and bounded to + /// 20 findings. Deliberately not shared with the CLI renderer, whose bytes + /// are golden-locked and must not move when this string is tuned. + /// + private static string RenderAudit(AuditReport report) + { + const int MaxResults = 20; + var sb = new StringBuilder(); + + sb.Append("as of: ").Append(report.AsOf.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture)).Append('\n'); + sb.Append("concepts: ").Append(report.ConceptCount).Append('\n'); + + sb.Append("\ntrust:\n"); + sb.Append($" {report.TrustCounts[TrustTier.HumanReviewed],4} human-reviewed\n"); + sb.Append($" {report.TrustCounts[TrustTier.MachineConfirmed],4} machine-confirmed\n"); + sb.Append($" {report.TrustCounts[TrustTier.Unverified],4} unverified\n"); + + sb.Append("\nstatus:\n"); + sb.Append($" {report.StatusCounts[ConceptStatus.Draft],4} draft\n"); + sb.Append($" {report.StatusCounts[ConceptStatus.Stable],4} stable\n"); + sb.Append($" {report.StatusCounts[ConceptStatus.Deprecated],4} deprecated\n"); + + sb.Append($"\nstale: {report.StaleCount} of {report.ConceptCount} past stale_after\n"); + + if (report.Findings.Count == 0) + { + sb.Append("\nneeds attention: none\n"); + return sb.ToString(); + } + + sb.Append($"\nneeds attention ({report.Findings.Count}):\n"); + foreach (var finding in report.Findings.Take(MaxResults)) + { + var freshness = finding.Lifecycle.StaleAfter is { } date + ? (finding.IsStale ? "stale " : "fresh ") + date.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture) + : "no-stale-after"; + + sb.Append(" ") + .Append(finding.Id) + .Append(" ").Append(freshness) + .Append(" ").Append(AuditVocabulary.Name(finding.Trust)) + .Append(" ").Append(AuditVocabulary.Name(finding.Lifecycle.Status)) + .Append('\n'); + } + + if (report.Findings.Count > MaxResults) + { + sb.Append($"… and {report.Findings.Count - MaxResults} more (narrow with stale/trust/status/type)\n"); + } + + return sb.ToString(); + } +``` + +d) Enregistrer le tool dans `GetTools()`, après `okf_search` : + +```csharp + AIFunctionFactory.Create(Audit, "okf_audit"), +``` + +e) Aucun `using` à ajouter : `System.ComponentModel`, `System.Globalization` et `System.Text` sont déjà en tête de `OkfBundleTools.cs`, et `ImplicitUsings` couvre `System.Linq`. + +- [ ] **Step 4: Lancer les tests pour vérifier qu'ils passent** + +Run: `dotnet test OKF4net.sln --filter "FullyQualifiedName~OKF4net.Tests.Agents"` +Expected: PASS, y compris `WriteToolNames_matches_the_three_mutating_tools_and_filters_them_out` (le nouveau tool étant en lecture seule, il doit apparaître dans le sous-ensemble read-only). Puis `dotnet test OKF4net.sln` en entier. + +- [ ] **Step 5: Commit** + +```bash +git add src/OKF4net.Agents/OkfBundleTools.cs tests/OKF4net.Tests/Agents/OkfAuditToolTests.cs +git commit -m "feat(agents): expose okf_audit as a read-only tool" +``` + +--- + +### Task 6: Documentation + +**Files:** +- Modify: `README.md` +- Modify: `CHANGELOG.md` + +**Interfaces:** +- Consumes: tout ce qui précède. +- Produces: rien. + +- [ ] **Step 1: Mettre à jour le README** + +Dans la liste des verbes CLI, ajouter `audit` juste après `validate`, avec un exemple qui montre la question de départ : + +````markdown +`okf audit ` reports trust, freshness and lifecycle across a whole +bundle: counts per trust tier (§5.3) and status (§5.4), plus the worklist of +stale concepts (§5.5). Filter it to ask corpus-level questions — + +```sh +# Which concepts are past stale_after and were never verified by a human? +okf audit bundles/acme_retail --stale --trust unverified,machine-confirmed +``` + +Without filter flags it selects exactly what `--stale` selects and prints the +summary form; with any filter flag it prints one line per matching concept, so +the output pipes. `--json` always emits the full document. Note the counts +always cover the whole bundle while `findings` covers the selection: `audit` is +a worklist, not an inventory (use `okf info --json` for that). +```` + +Dans la table « spec section → type », ajouter la ligne : + +```markdown +| §5.3–§5.5 | `ConceptAudit`, `AuditQuery`, `AuditReport` | Corpus-level trust/freshness query behind `okf audit` and `okf_audit` | +``` + +- [ ] **Step 2: Mettre à jour le CHANGELOG** + +Sous `## [Unreleased]`, section `### Added` (la créer si absente) : + +```markdown +- `okf audit` — a corpus-level query over a bundle's trust (§5.3), lifecycle + (§5.4) and staleness (§5.5) signals: counts plus a filterable worklist, with + `--stale`, `--trust`, `--status`, `--type`, `--as-of` and `--json`. Backed by + the new `ConceptAudit` in the core library and exposed to agents as the + read-only `okf_audit` tool. +``` + +- [ ] **Step 3: Formater et lancer la suite complète** + +```bash +dotnet format OKF4net.sln +dotnet build OKF4net.sln +dotnet test OKF4net.sln +``` + +Expected: build sans warning (ils sont des erreurs), 912 tests de base + les nouveaux, 0 échec, et `dotnet format --verify-no-changes` propre. + +- [ ] **Step 4: Commit** + +```bash +git add README.md CHANGELOG.md +git commit -m "docs(audit): document the okf audit verb and okf_audit tool" +``` + +--- + +## Reste à faire après le merge de la branche viewer + +`ROADMAP.md` et `CLAUDE.md` sont modifiés en parallèle par une autre session ; les toucher ici garantit un conflit. Une fois cette branche mergée : + +- `ROADMAP.md` : consigner l'échelle de l'article (dossier → index → recherche → graphe → bibliothèque fédérée) et marquer `okf audit` comme premier barreau livré. +- `CLAUDE.md` : une ligne sur `ConceptAudit` comme surface de requête unique partagée CLI/Agents, au même titre que la note « ne pas forker `ConceptSearch` ». diff --git a/docs/superpowers/specs/2026-08-21-okf-audit-design.md b/docs/superpowers/specs/2026-08-21-okf-audit-design.md index 22710b8..8a009ce 100644 --- a/docs/superpowers/specs/2026-08-21-okf-audit-design.md +++ b/docs/superpowers/specs/2026-08-21-okf-audit-design.md @@ -204,8 +204,15 @@ Règles de parsing des valeurs, pour lever toute ambiguïté : **Piège de parsing à ne pas rater.** `--trust`, `--status`, `--type` et `--as-of` consomment le token suivant : ils doivent être déclarés dans les `valuedFlags` de `Positional(args, "", ...)`, sinon `okf audit --as-of 2099-06-01 mon/bundle` -prendrait `2099-06-01` pour le chemin du bundle. C'est exactement ce que `--out` -a résolu pour `render` ([OkfCli.cs:130](../../../src/OKF4net.Cli/OkfCli.cs#L130)). +prendrait `2099-06-01` pour le chemin du bundle. + +**Dépendance de base à vérifier avant de commencer.** Ce mécanisme +(`Positional(…, valuedFlags)`) et le helper `FlagValue` n'existent **pas** sur +`origin/main` : ils ont été introduits par la branche +`okf-bundle-viewer-static-render` pour `render --out`, non mergée. Sur une base +qui ne les contient pas, `audit` doit les ajouter lui-même, en recopiant +verbatim l'implémentation de cette branche pour que le conflit au merge se +résolve à l'identique. Le plan d'implémentation en fait sa Task 0. ### 4.2 Deux modes de présentation, une seule sélection @@ -335,7 +342,9 @@ alignée sur les autres — le verbe occupe 8 colonnes, d'où quatre espaces apr et étendre la ligne d'option existante en `--json Machine-readable output for validate/info/audit`. Le commentaire -de classe de `OkfCli` (« Seven subcommands ») passe à huit et cite `audit`. +de classe de `OkfCli` énumère les sous-commandes et annonce leur nombre : +l'incrémenter en comptant les verbes réellement présents sur la base (six sur +`origin/main`, sept une fois le viewer mergé) et citer `audit`. ## 5. Unité 3 — tool agent `okf_audit` From 352ba31d940fd06d3f9b9e3063087e4612ba16cd Mon Sep 17 00:00:00 2001 From: Julien CHABLE Date: Fri, 21 Aug 2026 22:29:42 +0200 Subject: [PATCH 05/19] docs(audit): apply the external audit's findings to the plan and spec All four verified against the code first: - The tool bypassed RunTool, the guard every bundle-loading tool uses to turn OkfException into an "Error: ..." string. A bundle deleted after construction would have thrown out of a function tool instead of returning an error. Now wrapped, with a test that deletes the directory before the first load. - CmdAudit resolved the positional before validating flag values, so `okf audit --as-of` reported "missing " instead of naming the unvalued flag. Reordered, with a Theory over all four valued flags. - Both renderers spelled the trust/status labels as literals, which is precisely the drift AuditVocabulary was introduced to prevent. They now read every label from it; only the display ORDER stays local (the report shows the strongest tier first). - Task 1's interface block still advertised TrustTierNames/StatusNames, removed from the implementation during the previous review. Also folded in two design contracts the audit showed were missing from the spec rather than from the plan: why StalePolicy and AuditQuery.StaleOnly coexist (restitution vs worklist -- opposite questions), and why CLI input parses strictly while frontmatter parsing stays permissive (a typo absorbed into "stable" would make a worklist lie). Co-Authored-By: Claude Opus 5 (1M context) --- .../superpowers/plans/2026-08-21-okf-audit.md | 113 ++++++++++++++---- .../specs/2026-08-21-okf-audit-design.md | 30 ++++- 2 files changed, 121 insertions(+), 22 deletions(-) diff --git a/docs/superpowers/plans/2026-08-21-okf-audit.md b/docs/superpowers/plans/2026-08-21-okf-audit.md index d47e231..763a075 100644 --- a/docs/superpowers/plans/2026-08-21-okf-audit.md +++ b/docs/superpowers/plans/2026-08-21-okf-audit.md @@ -223,7 +223,7 @@ git commit -m "refactor(cli): add FlagValue and valued-flag-aware Positional" **Interfaces:** - Consumes: `Bundle.Concepts` (`IReadOnlyList`, chaque `Concept` porte `Id`, `Path`, `Document`), `Frontmatter.TrustTier`, `Frontmatter.Lifecycle`, `Frontmatter.Type`, `Frontmatter.Title`, `Lifecycle.IsStale(DateOnly)`, `Lifecycle.StaleAfter`, `Lifecycle.StaleAfterRaw`, `Lifecycle.Status`, `IOkfClock`/`SystemClock`. -- Produces: `ConceptAudit.Run(Bundle, AuditQuery, IOkfClock?) → AuditReport` ; `AuditQuery(bool StaleOnly, IReadOnlySet? Trust, ConceptStatus? Status, string? Type)` avec `AuditQuery.All` et `IsFiltered` ; `AuditFinding(ConceptId Id, string Path, string? Type, string? Title, TrustTier Trust, Lifecycle Lifecycle, bool IsStale)` ; `AuditReport` avec `AsOf`, `ConceptCount`, `TrustCounts`, `StatusCounts`, `StaleCount`, `Findings` ; `AuditVocabulary.Name(TrustTier)`, `Name(ConceptStatus)`, `TryParseTrustTier`, `TryParseStatus`, `TrustTierNames`, `StatusNames`. +- Produces: `ConceptAudit.Run(Bundle, AuditQuery, IOkfClock?) → AuditReport` ; `AuditQuery(bool StaleOnly, IReadOnlySet? Trust, ConceptStatus? Status, string? Type)` avec `AuditQuery.All` et `IsFiltered` ; `AuditFinding(ConceptId Id, string Path, string? Type, string? Title, TrustTier Trust, Lifecycle Lifecycle, bool IsStale)` ; `AuditReport` avec `AsOf`, `ConceptCount`, `TrustCounts`, `StatusCounts`, `StaleCount`, `Findings` ; `AuditVocabulary.Name(TrustTier)`, `Name(ConceptStatus)`, `TryParseTrustTier`, `TryParseStatus`, `TrustTiersInOrder`, `StatusesInOrder`. - [ ] **Step 1: Écrire les tests unitaires qui échouent** @@ -535,10 +535,18 @@ public sealed class AuditReport /// public static class AuditVocabulary { - /// The trust tiers, weakest to strongest -- the canonical serialization order. + /// + /// The trust tiers, weakest to strongest -- the canonical order. JSON + /// serializes in this order; the text report walks it in reverse, showing + /// the strongest tier first. + /// public static IReadOnlyList TrustTiersInOrder { get; } = [TrustTier.Unverified, TrustTier.MachineConfirmed, TrustTier.HumanReviewed]; + /// The lifecycle statuses in §5.4 order -- the order every surface displays them in. + public static IReadOnlyList StatusesInOrder { get; } = + [ConceptStatus.Draft, ConceptStatus.Stable, ConceptStatus.Deprecated]; + /// The wire/display name of a trust tier. public static string Name(TrustTier tier) => tier switch { @@ -819,6 +827,25 @@ Ajouter à `tests/OKF4net.Tests/CliTests.cs` (dans la classe `CliTests`) : Assert.Equal("", r.Err); } + /// + /// A valued flag left without a value must name itself, even when it is the + /// only argument -- otherwise the user is told the bundle is missing and the + /// real mistake is hidden. This is why CmdAudit validates flag values before + /// resolving the positional. + /// + [Theory] + [InlineData("--as-of")] + [InlineData("--trust")] + [InlineData("--status")] + [InlineData("--type")] + public void Audit_reports_a_valued_flag_left_without_a_value(string flag) + { + var r = Run("audit", flag); + + Assert.Equal(1, r.Code); + Assert.Equal($"error: {flag} requires a value\n", r.Err); + } + [Fact] public void Audit_as_of_alone_stays_in_report_mode() { @@ -891,7 +918,11 @@ c) Ajouter les membres privés : /// Implements the audit subcommand. private static int CmdAudit(string[] args, TextWriter stdout) { - var path = Positional(args, "", AuditValuedFlags); + // Flag values are validated BEFORE the positional is resolved. An + // unvalued flag is the more specific diagnosis, and `okf audit --as-of` + // -- the flag as the only argument -- would otherwise report + // "missing " and hide the actual mistake, because Positional + // skips a valued flag's slot without checking that it has a value. var clock = ParseAsOf(args); // Report mode selects exactly what --stale selects; only the @@ -899,6 +930,7 @@ c) Ajouter les membres privés : var filtered = AuditFilterFlags.Any(f => HasFlag(args, f)); var query = filtered ? ParseAuditQuery(args) : new AuditQuery(StaleOnly: true); + var path = Positional(args, "", AuditValuedFlags); var bundle = Load(path); var report = ConceptAudit.Run(bundle, query, clock); @@ -1000,15 +1032,22 @@ c) Ajouter les membres privés : stdout.Write($"as of: {report.AsOf.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture)}\n"); stdout.Write($"concepts: {report.ConceptCount}\n"); + // Labels always come from AuditVocabulary -- never as literals here. + // Duplicating them in each renderer is exactly the drift the shared + // vocabulary exists to prevent. Only the ORDER is decided locally: the + // report shows the strongest tier first, so it walks the canonical + // (weakest-first) list in reverse. stdout.Write("\ntrust:\n"); - stdout.Write($" {report.TrustCounts[TrustTier.HumanReviewed],4} human-reviewed\n"); - stdout.Write($" {report.TrustCounts[TrustTier.MachineConfirmed],4} machine-confirmed\n"); - stdout.Write($" {report.TrustCounts[TrustTier.Unverified],4} unverified\n"); + foreach (var tier in AuditVocabulary.TrustTiersInOrder.Reverse()) + { + stdout.Write($" {report.TrustCounts[tier],4} {AuditVocabulary.Name(tier)}\n"); + } stdout.Write("\nstatus:\n"); - stdout.Write($" {report.StatusCounts[ConceptStatus.Draft],4} draft\n"); - stdout.Write($" {report.StatusCounts[ConceptStatus.Stable],4} stable\n"); - stdout.Write($" {report.StatusCounts[ConceptStatus.Deprecated],4} deprecated\n"); + foreach (var status in AuditVocabulary.StatusesInOrder) + { + stdout.Write($" {report.StatusCounts[status],4} {AuditVocabulary.Name(status)}\n"); + } stdout.Write($"\nstale: {report.StaleCount} of {report.ConceptCount} past stale_after\n"); @@ -1467,6 +1506,24 @@ public class OkfAuditToolTests Assert.Contains("Usage: okf_audit", tools.Audit(status: "retired")); } + /// + /// A function tool returns errors, it does not throw them: a bundle that + /// disappears after the tool was constructed must surface as an "Error: ..." + /// string, which is what the shared RunTool guard provides. + /// + [Fact] + public void Audit_returns_an_error_string_when_the_bundle_cannot_be_loaded() + { + var tmp = new TempDir(); + tmp.Write("a.md", "---\ntype: Metric\n---\n"); + var tools = ToolsOver(tmp, new DateOnly(2026, 8, 21)); + tmp.Dispose(); // the directory is gone before the first load + + var text = tools.Audit(); + + Assert.StartsWith("Error: ", text); + } + [Fact] public void Audit_with_stale_false_and_no_filter_returns_the_whole_corpus() { @@ -1558,13 +1615,22 @@ c) Ajouter la méthode du tool : parsedStatus = value; } - var today = Today; - var report = ConceptAudit.Run( - GetBundle(), - new AuditQuery(stale, tiers, parsedStatus, type), - new PinnedClock(today)); + // Everything that can touch the filesystem goes through RunTool, the + // guard every bundle-loading tool uses: it turns OkfException (hence + // BundleLoadException), ArgumentException, IOException, + // UnauthorizedAccessException and DecoderFallbackException into an + // "Error: ..." string. A function tool must return an error, not throw + // one -- a directory deleted after construction would otherwise escape + // as an exception into the agent runtime. + return RunTool(() => + { + var report = ConceptAudit.Run( + GetBundle(), + new AuditQuery(stale, tiers, parsedStatus, type), + new PinnedClock(Today)); - return RenderAudit(report); + return RenderAudit(report); + }); } /// @@ -1581,15 +1647,20 @@ c) Ajouter la méthode du tool : sb.Append("as of: ").Append(report.AsOf.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture)).Append('\n'); sb.Append("concepts: ").Append(report.ConceptCount).Append('\n'); + // Same rule as the CLI renderer: labels from AuditVocabulary, never + // literals. The two renderers are separate on purpose (the CLI's bytes + // are golden-locked), but they must not spell the vocabulary twice. sb.Append("\ntrust:\n"); - sb.Append($" {report.TrustCounts[TrustTier.HumanReviewed],4} human-reviewed\n"); - sb.Append($" {report.TrustCounts[TrustTier.MachineConfirmed],4} machine-confirmed\n"); - sb.Append($" {report.TrustCounts[TrustTier.Unverified],4} unverified\n"); + foreach (var tier in AuditVocabulary.TrustTiersInOrder.Reverse()) + { + sb.Append($" {report.TrustCounts[tier],4} {AuditVocabulary.Name(tier)}\n"); + } sb.Append("\nstatus:\n"); - sb.Append($" {report.StatusCounts[ConceptStatus.Draft],4} draft\n"); - sb.Append($" {report.StatusCounts[ConceptStatus.Stable],4} stable\n"); - sb.Append($" {report.StatusCounts[ConceptStatus.Deprecated],4} deprecated\n"); + foreach (var status in AuditVocabulary.StatusesInOrder) + { + sb.Append($" {report.StatusCounts[status],4} {AuditVocabulary.Name(status)}\n"); + } sb.Append($"\nstale: {report.StaleCount} of {report.ConceptCount} past stale_after\n"); diff --git a/docs/superpowers/specs/2026-08-21-okf-audit-design.md b/docs/superpowers/specs/2026-08-21-okf-audit-design.md index 8a009ce..b792aee 100644 --- a/docs/superpowers/specs/2026-08-21-okf-audit-design.md +++ b/docs/superpowers/specs/2026-08-21-okf-audit-design.md @@ -151,6 +151,13 @@ public static class ConceptAudit indépendant de l'ordre de parcours du système de fichiers. - **Robustesse.** Aucune exception : errors-as-data comme le reste du cœur. Un bundle vide donne des compteurs à zéro et `Findings` vide. +- **Pourquoi pas `StalePolicy`.** Le cœur a déjà `StalePolicy` + (`Use`/`Tolerate(grace)`/`Strict`), que les Agents et le Catalog appliquent à + la *restitution*. Elle répond à « dois-je exposer ce concept à un + consommateur ? ». `AuditQuery.StaleOnly` répond à « ce concept est-il sur ma + liste de travail ? » — la question inverse, où un concept périmé est + précisément ce qu'on veut voir, pas ce qu'on veut filtrer. Les deux mécanismes + coexistent donc volontairement ; ne pas les fusionner. ## 4. Unité 2 — verbe CLI `okf audit` @@ -200,6 +207,18 @@ Règles de parsing des valeurs, pour lever toute ambiguïté : casse, puisque le spec ne contraint pas le vocabulaire de `type`. - Flag répété : la **première occurrence gagne**, comportement hérité de `FlagValue` (`Array.IndexOf`) et commun à tous les verbes existants. +- **Le parsing de l'entrée est strict, celui du frontmatter reste permissif.** + `Lifecycle.From` résout un `status` inconnu en `stable` (§5.4 : le chargement + ne rejette rien, §11), alors que `--status retired` doit échouer. Ce n'est pas + une incohérence : un producteur ne contrôle pas ce qu'il lit, un utilisateur + contrôle ce qu'il tape, et une faute de frappe silencieusement absorbée en + `stable` rendrait une worklist fausse. Les deux parsers restent donc distincts + — ne pas « harmoniser » le strict vers le permissif. +- **Ordre de validation.** Les valeurs des flags sont validées **avant** la + résolution du positionnel. Sinon `okf audit --as-of` (le flag comme unique + argument) rendrait `missing ` : `Positional` saute le créneau d'un flag + valué sans vérifier qu'il a une valeur, et le vrai défaut serait masqué par un + diagnostic moins précis. **Piège de parsing à ne pas rater.** `--trust`, `--status`, `--type` et `--as-of` consomment le token suivant : ils doivent être déclarés dans les `valuedFlags` de @@ -395,7 +414,16 @@ Différences assumées avec le CLI : que l'article défend ; - omet la ligne `bundle:` (le tool est lié à un seul bundle) ; - valeurs invalides de `trust`/`status` ⇒ message d'usage rendu comme chaîne (pas - d'exception), sur le modèle de `SearchUsageMessage`. + d'exception), sur le modèle de `SearchUsageMessage` ; +- **tout ce qui peut toucher le disque passe par `RunTool`**, la garde partagée + par tous les tools qui chargent le bundle : elle convertit `OkfException` + (donc `BundleLoadException`), `ArgumentException`, `IOException`, + `UnauthorizedAccessException` et `DecoderFallbackException` en une chaîne + `Error: …`. Un tool de fonction **rend** une erreur, il n'en lève pas : un + répertoire supprimé après la construction du tool remonterait sinon en + exception jusqu'au runtime de l'agent. Le rendu du vocabulaire suit la même + règle que le CLI — les libellés viennent d'`AuditVocabulary`, jamais de + littéraux recopiés. **Le rendu texte n'est pas partagé entre CLI et Agents** : seul le calcul (`ConceptAudit`) l'est. Raison : les octets du CLI sont verrouillés par des From 5406cbbbbf37184e1950a1104d516afa6b99757c Mon Sep 17 00:00:00 2001 From: Julien CHABLE Date: Fri, 21 Aug 2026 22:51:33 +0200 Subject: [PATCH 06/19] docs(audit): rebase the plan on dev, dropping the now-redundant Task 0 origin/dev carries the merged viewer work (PR #48), so FlagValue and Positional's valuedFlags parameter now exist: Task 0 added them and becomes a no-op, exactly as the plan anticipated. Removed it, restored the spec's `render --out` precedent, and corrected the subcommand count (seven here, eight with audit). Baseline after the merge: 987 tests, 0 failures. Co-Authored-By: Claude Opus 5 (1M context) --- .../superpowers/plans/2026-08-21-okf-audit.md | 182 ++---------------- .../specs/2026-08-21-okf-audit-design.md | 15 +- 2 files changed, 20 insertions(+), 177 deletions(-) diff --git a/docs/superpowers/plans/2026-08-21-okf-audit.md b/docs/superpowers/plans/2026-08-21-okf-audit.md index 763a075..b063301 100644 --- a/docs/superpowers/plans/2026-08-21-okf-audit.md +++ b/docs/superpowers/plans/2026-08-21-okf-audit.md @@ -18,28 +18,22 @@ - **Native AOT** : toute sérialisation JSON passe par `CliJsonContext` (source-generated) ; tout formatage/parsing de date utilise `CultureInfo.InvariantCulture` (le CLI est publié avec `InvariantGlobalization`). - **Fixtures** : ne jamais modifier un fichier existant sous `tests/fixtures/`. Ce plan n'ajoute que deux goldens neufs, écrits à la main. - **Aucune sortie existante ne change** : `validate`, `info`, `graph`, `parse`, `fmt`, `render` produisent les mêmes octets qu'avant. Si un golden existant casse, c'est une régression à corriger côté code. -- **Vérification** : `dotnet build OKF4net.sln` (warnings = erreurs) et `dotnet test OKF4net.sln` doivent être verts avant chaque commit. Baseline au départ de cette branche : **912 tests, 0 échec**. +- **Vérification** : `dotnet build OKF4net.sln` (warnings = erreurs) et `dotnet test OKF4net.sln` doivent être verts avant chaque commit. Baseline au départ de cette branche : **987 tests, 0 échec**. - **Format** : `dotnet format OKF4net.sln` avant le dernier commit (la CI lance `--verify-no-changes`). -## Base de la branche — à lire avant de commencer +## Base de la branche -Cette branche part d'`origin/main`, où le CLI a **six** sous-commandes -(`validate`, `info`, `index`, `graph`, `parse`, `fmt`). Le verbe `render` et le -projet `OKF4net.Viewer` vivent sur la branche `okf-bundle-viewer-static-render`, -non mergée. +`origin/dev` a été mergé dans cette branche le 2026-08-21 : le travail viewer +(PR #48) y est, donc le CLI a **sept** sous-commandes (`validate`, `info`, +`index`, `graph`, `parse`, `fmt`, `render`) et le projet `OKF4net.Viewer` existe. -Conséquence concrète : **les deux helpers de parsing que la spec présente comme -existants n'existent pas sur cette base.** `FlagValue` est absent, et -`Positional` n'a pas de paramètre `valuedFlags` — les deux ont été introduits par -la branche viewer pour `render --out`, que la spec §4.1 cite comme précédent. -C'est l'objet de la Task 0. +Ce merge a rendu caduque la Task 0 de la version précédente de ce plan, qui +ajoutait `FlagValue` et le paramètre `valuedFlags` de `Positional` : **les deux +existent désormais**, introduits par la branche viewer pour `render --out`. Ils +sont à utiliser tels quels, sans les modifier — la spec §4.1 les décrit comme le +précédent à suivre, ce qu'ils sont redevenus. -Choix retenu : rester sur `origin/main` pour que `audit` soit mergeable -indépendamment du viewer, et **recopier verbatim** l'implémentation de la branche -viewer plutôt que d'en écrire une variante. Les deux branches introduiront donc -le même code ; le conflit au merge se résout en gardant une seule copie, à -l'identique des deux côtés. Ne pas « améliorer » ces deux helpers ici : toute -divergence transformerait une résolution triviale en arbitrage. +Baseline après merge : **987 tests, 0 échec**. ## Écarts assumés par rapport à la spec @@ -52,7 +46,6 @@ Deux points où ce plan précise la spec plutôt que de la suivre à la lettre | Fichier | Rôle | Task | |---|---|---| -| `src/OKF4net.Cli/OkfCli.cs` (modifié) | `FlagValue` + `Positional(…, valuedFlags)` — prérequis de parsing | 0 | | `src/OKF4net/Audit.cs` (créé) | `AuditQuery`, `AuditFinding`, `AuditReport`, `ConceptAudit`, `AuditVocabulary` — le calcul et le vocabulaire partagés | 1 | | `tests/OKF4net.Tests/AuditTests.cs` (créé) | Tests unitaires du cœur | 1 | | `src/OKF4net.Cli/OkfCli.cs` (modifié) | Usage, dispatch, `CmdAudit`, parsing des flags, rendu texte | 2 | @@ -70,151 +63,6 @@ Deux points où ce plan précise la spec plutôt que de la suivre à la lettre --- -### Task 0: Les helpers de parsing prérequis - -**Files:** -- Modify: `src/OKF4net.Cli/OkfCli.cs` -- Test: `tests/OKF4net.Tests/CliTests.cs` - -**Interfaces:** -- Consumes: rien. -- Produces: `FlagValue(string[] args, string flag) → string?` (throw `CliOperationException` si le flag est présent sans valeur) et `Positional(string[] args, string what, params string[] valuedFlags)`. Les Tasks 2 et 3 en dépendent. - -**Code identique à la branche viewer — ne pas le réécrire autrement.** Si cette -branche est un jour rebasée sur une base qui contient déjà ces helpers, supprimer -purement et simplement cette task : elle devient un no-op. - -- [ ] **Step 1: Écrire les tests qui échouent** - -Ajouter à `tests/OKF4net.Tests/CliTests.cs` : - -```csharp - /// - /// A valued flag's value must never be mistaken for the positional - /// argument. Without the valuedFlags declaration, `--as-of` placed - /// before the bundle path swallows the path's slot. - /// - [Fact] - public void Valued_flag_value_is_not_taken_for_the_positional() - { - // `fmt` takes a file positional; the flag here is unknown to it, which - // is fine: the point is which token Positional returns. - var r = Run("fmt", "--as-of", "2099-06-01", Path.Combine(BundlePath, "tables", "users.md")); - - Assert.Equal(0, r.Code); - } - - [Fact] - public void Missing_positional_reports_the_expected_error() - { - var r = Run("validate"); - - Assert.Equal(1, r.Code); - Assert.Equal("error: missing \n", r.Err); - } -``` - -Le premier test échouera tant que `fmt` ne déclare pas `--as-of` en `valuedFlags` -— ce n'est pas son rôle. **Utiliser plutôt ce test une fois `audit` livré** -(Task 2 le couvre déjà avec la `[Theory]` sur les quatre flags valués). Pour la -Task 0, se limiter au second test plus à une vérification de compilation : les -helpers sont du code d'infrastructure dont les Tasks 2 et 3 sont les vrais -consommateurs, et leur couverture réelle arrive là. - -- [ ] **Step 2: Lancer le test pour vérifier l'état de départ** - -Run: `dotnet test OKF4net.sln --filter "FullyQualifiedName~CliTests.Missing_positional"` -Expected: PASS (comportement déjà présent) — il sert de garde-fou pendant la modification de `Positional`. - -- [ ] **Step 3: Ajouter les helpers** - -Dans `src/OKF4net.Cli/OkfCli.cs`, remplacer la signature et le corps de `Positional` : - -```csharp - /// - /// Returns the first positional argument, or throws. Everything after a - /// -- separator is treated as positional (so paths beginning with - /// - work). - /// - /// The command's argument list. - /// Description of the missing positional, used in the error message. - /// - /// Flags that consume the following token as their value (e.g. --as-of) - /// rather than as a candidate positional. Verbs whose flags are all valueless - /// pass nothing, and the scan behaves exactly as before. - /// - private static string Positional(string[] args, string what, params string[] valuedFlags) - { - var sepIdx = Array.IndexOf(args, "--"); - if (sepIdx >= 0 && sepIdx + 1 < args.Length) - { - return args[sepIdx + 1]; - } - - for (var i = 0; i < args.Length; i++) - { - var a = args[i]; - if (Array.IndexOf(valuedFlags, a) >= 0) - { - i++; // Skip the value that belongs to this flag. - continue; - } - - if (!a.StartsWith('-')) - { - return a; - } - } - - throw new CliOperationException($"missing {what}"); - } -``` - -et ajouter, juste après `HasFlag` : - -```csharp - /// - /// The value following , or null when the - /// flag is absent. Throws when the flag is present but unvalued. - /// - private static string? FlagValue(string[] args, string flag) - { - var index = Array.IndexOf(args, flag); - if (index < 0) - { - return null; - } - - if (index + 1 >= args.Length) - { - throw new CliOperationException($"{flag} requires a value"); - } - - return args[index + 1]; - } -``` - -`FlagValue` n'a encore aucun appelant à ce stade. C'est volontaire : la Task 2 -est son consommateur. Le compilateur ne s'en plaint pas (une méthode privée -inutilisée ne produit pas de warning dans ce projet ; si un analyseur devait le -signaler, fusionner cette task avec la Task 2 plutôt que d'ajouter une -suppression). - -- [ ] **Step 4: Vérifier la non-régression** - -Run: `dotnet test OKF4net.sln` -Expected: 912 tests, 0 échec — aucun verbe existant ne change de comportement, -`valuedFlags` étant vide pour tous. - -- [ ] **Step 5: Commit** - -```bash -git add src/OKF4net.Cli/OkfCli.cs tests/OKF4net.Tests/CliTests.cs -git commit -m "refactor(cli): add FlagValue and valued-flag-aware Positional" -``` - ---- - ### Task 1: Le calcul partagé (`ConceptAudit`) **Files:** @@ -676,7 +524,7 @@ public static class ConceptAudit Run: `dotnet test OKF4net.sln --filter "FullyQualifiedName~AuditTests"` Expected: PASS — 12 méthodes de test, 13 cas exécutés (la `[Theory]` en compte -deux). Puis `dotnet test OKF4net.sln` en entier : 912 tests de base + les +deux). Puis `dotnet test OKF4net.sln` en entier : 987 tests de base + les nouveaux, 0 échec. - [ ] **Step 5: Commit** @@ -890,9 +738,7 @@ et remplacer la ligne d'option `--json` par : ``` Mettre aussi à jour le commentaire XML de la classe : sur cette base il annonce -« Six subcommands » — il passe à sept, en citant `audit`. (Sur une base où le -viewer est mergé, ce serait sept → huit : compter les verbes réellement présents -plutôt que recopier ce chiffre.) +« Seven subcommands » — il passe à huit, en citant `audit`. b) Dans le `switch` de `Run`, ajouter après `"validate"` : @@ -1773,7 +1619,7 @@ dotnet build OKF4net.sln dotnet test OKF4net.sln ``` -Expected: build sans warning (ils sont des erreurs), 912 tests de base + les nouveaux, 0 échec, et `dotnet format --verify-no-changes` propre. +Expected: build sans warning (ils sont des erreurs), 987 tests de base + les nouveaux, 0 échec, et `dotnet format --verify-no-changes` propre. - [ ] **Step 4: Commit** diff --git a/docs/superpowers/specs/2026-08-21-okf-audit-design.md b/docs/superpowers/specs/2026-08-21-okf-audit-design.md index b792aee..e0d7981 100644 --- a/docs/superpowers/specs/2026-08-21-okf-audit-design.md +++ b/docs/superpowers/specs/2026-08-21-okf-audit-design.md @@ -225,13 +225,10 @@ consomment le token suivant : ils doivent être déclarés dans les `valuedFlags `Positional(args, "", ...)`, sinon `okf audit --as-of 2099-06-01 mon/bundle` prendrait `2099-06-01` pour le chemin du bundle. -**Dépendance de base à vérifier avant de commencer.** Ce mécanisme -(`Positional(…, valuedFlags)`) et le helper `FlagValue` n'existent **pas** sur -`origin/main` : ils ont été introduits par la branche -`okf-bundle-viewer-static-render` pour `render --out`, non mergée. Sur une base -qui ne les contient pas, `audit` doit les ajouter lui-même, en recopiant -verbatim l'implémentation de cette branche pour que le conflit au merge se -résolve à l'identique. Le plan d'implémentation en fait sa Task 0. +C'est exactement ce que `--out` a résolu pour `render`. Note de base : ces deux +helpers (`Positional(…, valuedFlags)` et `FlagValue`) sont arrivés avec le +travail viewer ; ils sont présents depuis le merge d'`origin/dev` dans la branche +d'implémentation. Sur une base antérieure, il faudrait les ajouter d'abord. ### 4.2 Deux modes de présentation, une seule sélection @@ -362,8 +359,8 @@ alignée sur les autres — le verbe occupe 8 colonnes, d'où quatre espaces apr et étendre la ligne d'option existante en `--json Machine-readable output for validate/info/audit`. Le commentaire de classe de `OkfCli` énumère les sous-commandes et annonce leur nombre : -l'incrémenter en comptant les verbes réellement présents sur la base (six sur -`origin/main`, sept une fois le viewer mergé) et citer `audit`. +l'incrémenter en comptant les verbes réellement présents sur la base (sept +depuis le merge du viewer, donc huit avec `audit`) et citer `audit`. ## 5. Unité 3 — tool agent `okf_audit` From 53a82e0fc49fcc3abc9ee8cf399e763e7e8b0993 Mon Sep 17 00:00:00 2001 From: Julien CHABLE Date: Fri, 21 Aug 2026 22:55:11 +0200 Subject: [PATCH 07/19] feat(audit): add ConceptAudit, the shared corpus-level query --- src/OKF4net/Audit.cs | 231 ++++++++++++++++++++++++++++++ tests/OKF4net.Tests/AuditTests.cs | 199 +++++++++++++++++++++++++ 2 files changed, 430 insertions(+) create mode 100644 src/OKF4net/Audit.cs create mode 100644 tests/OKF4net.Tests/AuditTests.cs diff --git a/src/OKF4net/Audit.cs b/src/OKF4net/Audit.cs new file mode 100644 index 0000000..6685d95 --- /dev/null +++ b/src/OKF4net/Audit.cs @@ -0,0 +1,231 @@ +// SPDX-License-Identifier: LGPL-3.0-or-later +namespace OKF4net; + +/// +/// The selection predicates of an audit (§5.3–§5.5). Predicates combine with +/// AND; default selects every concept. +/// +/// +/// The generated equality compares by reference (the +/// behaviour of EqualityComparer<IReadOnlySet<T>>.Default), so two +/// logically identical queries may compare unequal. Do not rely on it, and do +/// not use an as a dictionary key: the record struct is +/// for with and ToString, not for its equality. +/// +/// Keep only concepts past their stale_after date. +/// Keep only concepts whose derived tier is in this set; null keeps every tier. +/// Keep only concepts with this lifecycle status; null keeps every status. +/// Keep only concepts whose frontmatter type matches exactly (ordinal); null keeps every type. +public readonly record struct AuditQuery( + bool StaleOnly = false, + IReadOnlySet? Trust = null, + ConceptStatus? Status = null, + string? Type = null) +{ + /// The query that keeps every concept. + public static AuditQuery All => default; + + /// True as soon as one predicate is set. + public bool IsFiltered => StaleOnly || Trust is not null || Status is not null || Type is not null; +} + +/// One concept selected by an audit, with its signals already derived. +/// The concept id (§2). +/// The concept's file path, as built from the bundle root. +/// The frontmatter type, or null when absent. +/// The frontmatter title, or null when absent. +/// The derived trust tier (§5.3). +/// The lifecycle fields (§5.4/§5.5). +/// Whether the concept is stale as of the report's . +public readonly record struct AuditFinding( + ConceptId Id, + string Path, + string? Type, + string? Title, + TrustTier Trust, + Lifecycle Lifecycle, + bool IsStale); + +/// +/// The result of an audit: counts over the whole bundle, plus the concepts the +/// query selected. The counts never narrow with the query -- the denominator +/// stays stable while moves. +/// +public sealed class AuditReport +{ + internal AuditReport( + DateOnly asOf, + int conceptCount, + IReadOnlyDictionary trustCounts, + IReadOnlyDictionary statusCounts, + int staleCount, + IReadOnlyList findings) + { + AsOf = asOf; + ConceptCount = conceptCount; + TrustCounts = trustCounts; + StatusCounts = statusCounts; + StaleCount = staleCount; + Findings = findings; + } + + /// The observation date staleness was evaluated against. + public DateOnly AsOf { get; } + + /// The number of concepts in the bundle (not in the selection). + public int ConceptCount { get; } + + /// Concept counts per trust tier over the whole bundle; all three keys are always present. + public IReadOnlyDictionary TrustCounts { get; } + + /// Concept counts per lifecycle status over the whole bundle; all three keys are always present. + public IReadOnlyDictionary StatusCounts { get; } + + /// The number of stale concepts in the whole bundle. + public int StaleCount { get; } + + /// The selected concepts, sorted by concept id (ordinal). + public IReadOnlyList Findings { get; } +} + +/// +/// The single spelling of the audit vocabularies, shared by every surface (CLI +/// input, CLI text, JSON, agent tool) so no two layers can drift apart. +/// +public static class AuditVocabulary +{ + /// + /// The trust tiers, weakest to strongest -- the canonical order. JSON + /// serializes in this order; the text report walks it in reverse, showing + /// the strongest tier first. + /// + public static IReadOnlyList TrustTiersInOrder { get; } = + [TrustTier.Unverified, TrustTier.MachineConfirmed, TrustTier.HumanReviewed]; + + /// The lifecycle statuses in §5.4 order -- the order every surface displays them in. + public static IReadOnlyList StatusesInOrder { get; } = + [ConceptStatus.Draft, ConceptStatus.Stable, ConceptStatus.Deprecated]; + + /// The wire/display name of a trust tier. + public static string Name(TrustTier tier) => tier switch + { + TrustTier.HumanReviewed => "human-reviewed", + TrustTier.MachineConfirmed => "machine-confirmed", + _ => "unverified", + }; + + /// The wire/display name of a lifecycle status. + public static string Name(ConceptStatus status) => status switch + { + ConceptStatus.Draft => "draft", + ConceptStatus.Deprecated => "deprecated", + _ => "stable", + }; + + /// Parses a trust tier name (exact, ordinal). Unlike frontmatter parsing, an unknown name fails rather than defaulting. + public static bool TryParseTrustTier(string text, out TrustTier tier) + { + switch (text) + { + case "unverified": tier = TrustTier.Unverified; return true; + case "machine-confirmed": tier = TrustTier.MachineConfirmed; return true; + case "human-reviewed": tier = TrustTier.HumanReviewed; return true; + default: tier = TrustTier.Unverified; return false; + } + } + + /// Parses a lifecycle status name (exact, ordinal). Unlike , an unknown name fails rather than resolving to stable. + public static bool TryParseStatus(string text, out ConceptStatus status) + { + switch (text) + { + case "draft": status = ConceptStatus.Draft; return true; + case "stable": status = ConceptStatus.Stable; return true; + case "deprecated": status = ConceptStatus.Deprecated; return true; + default: status = ConceptStatus.Stable; return false; + } + } +} + +/// +/// Queries a bundle's §5.3–§5.5 signals. Reads nothing from disk, writes +/// nothing, and never throws on data: an unparseable document is already absent +/// from , and a malformed stale_after simply +/// reads as "not stale" (the validator owns that diagnostic). +/// +public static class ConceptAudit +{ + /// Runs over . + /// The loaded bundle. + /// The selection predicates; default selects everything. + /// Supplies "today" for staleness (§5.5); defaults to . + public static AuditReport Run(Bundle bundle, AuditQuery query = default, IOkfClock? clock = null) + { + var asOf = (clock ?? new SystemClock()).Today; + + var trustCounts = new Dictionary + { + [TrustTier.Unverified] = 0, + [TrustTier.MachineConfirmed] = 0, + [TrustTier.HumanReviewed] = 0, + }; + + var statusCounts = new Dictionary + { + [ConceptStatus.Draft] = 0, + [ConceptStatus.Stable] = 0, + [ConceptStatus.Deprecated] = 0, + }; + + var staleCount = 0; + var findings = new List(); + + foreach (var concept in bundle.Concepts) + { + var frontmatter = concept.Document.Frontmatter; + var tier = frontmatter.TrustTier; + var lifecycle = frontmatter.Lifecycle; + var isStale = lifecycle.IsStale(asOf); + + trustCounts[tier]++; + statusCounts[lifecycle.Status]++; + if (isStale) + { + staleCount++; + } + + if (query.StaleOnly && !isStale) + { + continue; + } + + if (query.Trust is { } tiers && !tiers.Contains(tier)) + { + continue; + } + + if (query.Status is { } status && lifecycle.Status != status) + { + continue; + } + + if (query.Type is { } type && !string.Equals(frontmatter.Type, type, StringComparison.Ordinal)) + { + continue; + } + + findings.Add(new AuditFinding( + concept.Id, + concept.Path, + frontmatter.Type, + frontmatter.Title, + tier, + lifecycle, + isStale)); + } + + findings.Sort(static (a, b) => string.CompareOrdinal(a.Id.ToString(), b.Id.ToString())); + + return new AuditReport(asOf, bundle.Count, trustCounts, statusCounts, staleCount, findings); + } +} diff --git a/tests/OKF4net.Tests/AuditTests.cs b/tests/OKF4net.Tests/AuditTests.cs new file mode 100644 index 0000000..f9d5981 --- /dev/null +++ b/tests/OKF4net.Tests/AuditTests.cs @@ -0,0 +1,199 @@ +// SPDX-License-Identifier: LGPL-3.0-or-later +namespace OKF4net.Tests; + +/// +/// Unit tests for : tier and status counting, +/// the §5.5 staleness boundary, predicate composition and ordering. +/// Every test pins the date with so nothing here +/// depends on the day the suite runs. +/// +public class AuditTests +{ + private static readonly DateOnly Today = new(2026, 8, 21); + + private static Bundle Load(TempDir tmp) => Bundle.Load(tmp.Path); + + private static AuditReport Audit(TempDir tmp, AuditQuery query = default) + => ConceptAudit.Run(Load(tmp), query, new FixedClock(Today)); + + [Fact] + public void Counts_the_three_trust_tiers() + { + using var tmp = new TempDir(); + tmp.Write("a.md", "---\ntype: Metric\nverified:\n - { by: human:ada, at: 2026-01-01T00:00:00Z }\n---\n"); + tmp.Write("b.md", "---\ntype: Metric\nverified:\n - { by: process:nightly, at: 2026-01-01T00:00:00Z }\n---\n"); + tmp.Write("c.md", "---\ntype: Metric\n---\n"); + + var report = Audit(tmp); + + Assert.Equal(3, report.ConceptCount); + Assert.Equal(1, report.TrustCounts[TrustTier.HumanReviewed]); + Assert.Equal(1, report.TrustCounts[TrustTier.MachineConfirmed]); + Assert.Equal(1, report.TrustCounts[TrustTier.Unverified]); + } + + [Fact] + public void Unknown_status_counts_as_stable() + { + using var tmp = new TempDir(); + tmp.Write("a.md", "---\ntype: Metric\nstatus: retired\n---\n"); + tmp.Write("b.md", "---\ntype: Metric\nstatus: draft\n---\n"); + + var report = Audit(tmp); + + Assert.Equal(1, report.StatusCounts[ConceptStatus.Stable]); + Assert.Equal(1, report.StatusCounts[ConceptStatus.Draft]); + Assert.Equal(0, report.StatusCounts[ConceptStatus.Deprecated]); + } + + [Theory] + [InlineData("2026-08-21", true)] // §5.5: today >= stale_after -- the exact boundary IS stale. + [InlineData("2026-08-22", false)] + public void Staleness_boundary_follows_section_5_5(string staleAfter, bool expectedStale) + { + using var tmp = new TempDir(); + tmp.Write("a.md", $"---\ntype: Metric\nstale_after: {staleAfter}\n---\n"); + + // The default query filters nothing, so the single concept is always returned. + var report = Audit(tmp); + + Assert.Equal(expectedStale, report.Findings.Single().IsStale); + Assert.Equal(expectedStale ? 1 : 0, report.StaleCount); + } + + [Fact] + public void Malformed_or_absent_stale_after_is_never_stale() + { + using var tmp = new TempDir(); + tmp.Write("bad.md", "---\ntype: Metric\nstale_after: not-a-date\n---\n"); + tmp.Write("none.md", "---\ntype: Metric\n---\n"); + + var report = Audit(tmp); + + Assert.Equal(0, report.StaleCount); + Assert.Empty(ConceptAudit.Run(Load(tmp), new AuditQuery(StaleOnly: true), new FixedClock(Today)).Findings); + Assert.True(report.Findings.Single(f => f.Id.ToString() == "bad").Lifecycle.StaleAfterMalformed); + } + + [Fact] + public void Predicates_compose_with_and() + { + using var tmp = new TempDir(); + tmp.Write("stale-unverified.md", "---\ntype: Metric\nstale_after: 2026-01-01\n---\n"); + tmp.Write("stale-human.md", "---\ntype: Metric\nstale_after: 2026-01-01\nverified:\n - { by: human:ada, at: 2026-01-01T00:00:00Z }\n---\n"); + tmp.Write("fresh-unverified.md", "---\ntype: Metric\nstale_after: 2099-01-01\n---\n"); + + var query = new AuditQuery( + StaleOnly: true, + Trust: new HashSet { TrustTier.Unverified }); + + var findings = ConceptAudit.Run(Load(tmp), query, new FixedClock(Today)).Findings; + + Assert.Equal(["stale-unverified"], findings.Select(f => f.Id.ToString())); + } + + [Fact] + public void Findings_are_sorted_by_concept_id_ordinal() + { + using var tmp = new TempDir(); + tmp.Write("zeta.md", "---\ntype: Metric\n---\n"); + tmp.Write("alpha.md", "---\ntype: Metric\n---\n"); + tmp.Write("mid/beta.md", "---\ntype: Metric\n---\n"); + + var findings = Audit(tmp).Findings.Select(f => f.Id.ToString()).ToList(); + + Assert.Equal(["alpha", "mid/beta", "zeta"], findings); + } + + [Fact] + public void Counts_cover_the_whole_bundle_even_when_the_query_filters() + { + using var tmp = new TempDir(); + tmp.Write("stale.md", "---\ntype: Metric\nstale_after: 2026-01-01\n---\n"); + tmp.Write("fresh.md", "---\ntype: Metric\n---\n"); + + var report = ConceptAudit.Run(Load(tmp), new AuditQuery(StaleOnly: true), new FixedClock(Today)); + + Assert.Single(report.Findings); + Assert.Equal(2, report.ConceptCount); + Assert.Equal(2, report.TrustCounts[TrustTier.Unverified]); + Assert.Equal(1, report.StaleCount); + } + + [Fact] + public void Empty_bundle_yields_zeroed_counts_and_no_findings() + { + using var tmp = new TempDir(); + + var report = Audit(tmp); + + Assert.Equal(0, report.ConceptCount); + Assert.Empty(report.Findings); + Assert.Equal(0, report.TrustCounts[TrustTier.HumanReviewed]); + Assert.Equal(0, report.StatusCounts[ConceptStatus.Deprecated]); + } + + /// + /// A document whose frontmatter cannot be parsed lands in + /// Bundle.ParseErrors (permissive loading) and is not a concept, so + /// it must not reach any counter. Note this is a *parse* failure -- a truly + /// unreadable file (I/O, permissions, non-UTF-8) throws + /// BundleLoadException and never reaches . + /// + [Fact] + public void Unparseable_documents_are_excluded_from_every_count() + { + using var tmp = new TempDir(); + tmp.Write("ok.md", "---\ntype: Metric\n---\n"); + tmp.Write("broken.md", "---\ntype: Metric\n"); // unterminated frontmatter block + + var bundle = Load(tmp); + Assert.Single(bundle.ParseErrors); + + var report = ConceptAudit.Run(bundle, default, new FixedClock(Today)); + + Assert.Equal(1, report.ConceptCount); + Assert.Single(report.Findings); + } + + [Fact] + public void Null_clock_falls_back_to_today_in_utc() + { + using var tmp = new TempDir(); + tmp.Write("a.md", "---\ntype: Metric\n---\n"); + + var report = ConceptAudit.Run(Load(tmp)); + + Assert.Equal(DateOnly.FromDateTime(DateTime.UtcNow.Date), report.AsOf); + } + + [Fact] + public void Type_filter_is_exact_and_ordinal() + { + using var tmp = new TempDir(); + tmp.Write("metric.md", "---\ntype: Metric\n---\n"); + tmp.Write("lower.md", "---\ntype: metric\n---\n"); + tmp.Write("untyped.md", "---\ntitle: No type here\n---\n"); + + var findings = ConceptAudit.Run(Load(tmp), new AuditQuery(Type: "Metric"), new FixedClock(Today)).Findings; + + Assert.Equal(["metric"], findings.Select(f => f.Id.ToString())); + } + + [Fact] + public void Vocabulary_names_round_trip() + { + Assert.Equal("human-reviewed", AuditVocabulary.Name(TrustTier.HumanReviewed)); + Assert.Equal("machine-confirmed", AuditVocabulary.Name(TrustTier.MachineConfirmed)); + Assert.Equal("unverified", AuditVocabulary.Name(TrustTier.Unverified)); + Assert.Equal("draft", AuditVocabulary.Name(ConceptStatus.Draft)); + + Assert.True(AuditVocabulary.TryParseTrustTier("machine-confirmed", out var tier)); + Assert.Equal(TrustTier.MachineConfirmed, tier); + Assert.False(AuditVocabulary.TryParseTrustTier("machine", out _)); + + Assert.True(AuditVocabulary.TryParseStatus("deprecated", out var status)); + Assert.Equal(ConceptStatus.Deprecated, status); + Assert.False(AuditVocabulary.TryParseStatus("retired", out _)); + } +} From 05677040e24af0c18e506431c82662d42324393d Mon Sep 17 00:00:00 2001 From: Julien CHABLE Date: Fri, 21 Aug 2026 23:01:58 +0200 Subject: [PATCH 08/19] fix(audit): use component-wise ConceptId ordering, add discriminating test case --- src/OKF4net/Audit.cs | 2 +- tests/OKF4net.Tests/AuditTests.cs | 10 ++++++++-- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/src/OKF4net/Audit.cs b/src/OKF4net/Audit.cs index 6685d95..ba14c7a 100644 --- a/src/OKF4net/Audit.cs +++ b/src/OKF4net/Audit.cs @@ -224,7 +224,7 @@ public static AuditReport Run(Bundle bundle, AuditQuery query = default, IOkfClo isStale)); } - findings.Sort(static (a, b) => string.CompareOrdinal(a.Id.ToString(), b.Id.ToString())); + findings.Sort(static (a, b) => a.Id.CompareTo(b.Id)); return new AuditReport(asOf, bundle.Count, trustCounts, statusCounts, staleCount, findings); } diff --git a/tests/OKF4net.Tests/AuditTests.cs b/tests/OKF4net.Tests/AuditTests.cs index f9d5981..f4874ee 100644 --- a/tests/OKF4net.Tests/AuditTests.cs +++ b/tests/OKF4net.Tests/AuditTests.cs @@ -93,16 +93,22 @@ public void Predicates_compose_with_and() } [Fact] - public void Findings_are_sorted_by_concept_id_ordinal() + public void Findings_are_sorted_by_concept_id_component_wise() { using var tmp = new TempDir(); tmp.Write("zeta.md", "---\ntype: Metric\n---\n"); tmp.Write("alpha.md", "---\ntype: Metric\n---\n"); tmp.Write("mid/beta.md", "---\ntype: Metric\n---\n"); + // Discriminating pair: component-wise ordering differs from flat ordinal. + // ConceptId.CompareTo compares segments; "orders/extra" has segments ["orders", "extra"], + // and "orders-extra" is a single segment. The first segment "orders" < "orders-extra", + // so "orders/extra" comes before "orders-extra" under component-wise ordering. + tmp.Write("orders-extra.md", "---\ntype: Metric\n---\n"); + tmp.Write("orders/extra.md", "---\ntype: Metric\n---\n"); var findings = Audit(tmp).Findings.Select(f => f.Id.ToString()).ToList(); - Assert.Equal(["alpha", "mid/beta", "zeta"], findings); + Assert.Equal(["alpha", "mid/beta", "orders/extra", "orders-extra", "zeta"], findings); } [Fact] From 1a2e845ab07bd15d7ddca5884311108bd61e0762 Mon Sep 17 00:00:00 2001 From: Julien CHABLE Date: Fri, 21 Aug 2026 23:09:36 +0200 Subject: [PATCH 09/19] feat(cli): add the okf audit verb with report, query and --json modes Adds `okf audit`, wired to the ConceptAudit engine shipped earlier: report mode (bundle-wide trust/status/staleness summary plus a worklist), query mode (--stale/--trust/--status/--type select a bare concept-line listing), and --json (source-generated, AOT-safe) for both. Implemented as one unit per the task-2-3 brief's controller ruling, which forbids the intermediate --json stub the original two-task split called for. --- src/OKF4net.Cli/JsonOutput.cs | 86 ++++++++++++ src/OKF4net.Cli/OkfCli.cs | 176 +++++++++++++++++++++++- tests/OKF4net.Tests/CliTests.cs | 228 ++++++++++++++++++++++++++++++++ 3 files changed, 485 insertions(+), 5 deletions(-) diff --git a/src/OKF4net.Cli/JsonOutput.cs b/src/OKF4net.Cli/JsonOutput.cs index 31b5668..aef0bef 100644 --- a/src/OKF4net.Cli/JsonOutput.cs +++ b/src/OKF4net.Cli/JsonOutput.cs @@ -1,4 +1,5 @@ // SPDX-License-Identifier: LGPL-3.0-or-later +using System.Globalization; using System.Text.Json; using System.Text.Json.Serialization; @@ -38,6 +39,37 @@ internal sealed record InfoJsonResult( int BrokenLinkCount, IReadOnlyList ParseErrors); +/// The query okf audit applied, replayed for --json consumers. +internal sealed record AuditQueryJson(bool Stale, IReadOnlyList? Trust, string? Status, string? Type); + +/// Concept counts per trust tier (§5.3), over the whole bundle. +internal sealed record TrustCountsJson(int HumanReviewed, int MachineConfirmed, int Unverified); + +/// Concept counts per lifecycle status (§5.4), over the whole bundle. +internal sealed record StatusCountsJson(int Draft, int Stable, int Deprecated); + +/// One selected concept, projected for --json output. +internal sealed record AuditFindingJson( + string ConceptId, + string Path, + string? Type, + string? Title, + string Trust, + string Status, + string? StaleAfter, + bool Stale); + +/// The full result of okf audit --json. +internal sealed record AuditJsonResult( + string Bundle, + string AsOf, + int ConceptCount, + AuditQueryJson Query, + TrustCountsJson Trust, + StatusCountsJson Status, + int StaleCount, + IReadOnlyList Findings); + /// /// Source-generated for every /// --json output type. Required, not optional, because the CLI is @@ -49,6 +81,7 @@ internal sealed record InfoJsonResult( [JsonSourceGenerationOptions(PropertyNamingPolicy = JsonKnownNamingPolicy.CamelCase)] [JsonSerializable(typeof(ValidateJsonResult))] [JsonSerializable(typeof(InfoJsonResult))] +[JsonSerializable(typeof(AuditJsonResult))] internal partial class CliJsonContext : JsonSerializerContext { } @@ -112,6 +145,59 @@ internal static void WriteInfo(TextWriter stdout, string bundlePath, Bundle bund stdout.Write("\n"); } + /// Writes okf audit --json's result to as a single line-terminated JSON document. + internal static void WriteAudit(TextWriter stdout, string bundlePath, AuditQuery query, AuditReport report) + { + // Serialized in ladder order, never in the order the user typed them: + // IReadOnlySet has no guaranteed order, and the document must be + // reproducible. Written as a statement rather than a ternary so the + // nullable analysis narrows `Trust` through the pattern -- it does not + // narrow a property across the arms of a conditional. + List? trustQuery = null; + if (query.Trust is { } selectedTiers) + { + trustQuery = AuditVocabulary.TrustTiersInOrder + .Where(selectedTiers.Contains) + .Select(AuditVocabulary.Name) + .ToList(); + } + + var findings = report.Findings + .Select(f => new AuditFindingJson( + f.Id.ToString(), + f.Path, + f.Type, + f.Title, + AuditVocabulary.Name(f.Trust), + AuditVocabulary.Name(f.Lifecycle.Status), + f.Lifecycle.StaleAfterRaw, + f.IsStale)) + .ToList(); + + var result = new AuditJsonResult( + bundlePath, + report.AsOf.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture), + report.ConceptCount, + new AuditQueryJson( + query.StaleOnly, + trustQuery, + query.Status is { } status ? AuditVocabulary.Name(status) : null, + query.Type), + new TrustCountsJson( + report.TrustCounts[TrustTier.HumanReviewed], + report.TrustCounts[TrustTier.MachineConfirmed], + report.TrustCounts[TrustTier.Unverified]), + new StatusCountsJson( + report.StatusCounts[ConceptStatus.Draft], + report.StatusCounts[ConceptStatus.Stable], + report.StatusCounts[ConceptStatus.Deprecated]), + report.StaleCount, + findings); + + stdout.Write(JsonSerializer.Serialize(result, CliJsonContext.Default.AuditJsonResult)); + stdout.Write("\n"); + } + /// Counts concepts by frontmatter type (a missing type counts as "(none)"), sorted ordinally by type name. internal static SortedDictionary BuildTypeHistogram(Bundle bundle) { diff --git a/src/OKF4net.Cli/OkfCli.cs b/src/OKF4net.Cli/OkfCli.cs index 301ef28..26c0fe1 100644 --- a/src/OKF4net.Cli/OkfCli.cs +++ b/src/OKF4net.Cli/OkfCli.cs @@ -1,4 +1,5 @@ // SPDX-License-Identifier: LGPL-3.0-or-later +using System.Globalization; using System.Text; using OKF4net.Internal; using OKF4net.Viewer; @@ -6,10 +7,10 @@ namespace OKF4net.Cli; /// -/// The okf command-line tool. Seven subcommands (validate, -/// info, index, graph, parse, fmt, -/// render) over hand-rolled argument parsing -- no third-party -/// dependencies. +/// The okf command-line tool. Eight subcommands (validate, +/// audit, info, index, graph, parse, +/// fmt, render) over hand-rolled argument parsing -- no +/// third-party dependencies. /// /// is the sole public entry point so tests can drive the /// CLI in-process (capturing stdout/stderr) without spawning a subprocess; @@ -29,6 +30,7 @@ public static class OkfCli "\n" + "COMMANDS:\n" + " validate Check a bundle against OKF v0.2 conformance (§11)\n" + + " audit Report trust, freshness and lifecycle across the bundle\n" + " info Summarize a bundle (concepts, types, links, version)\n" + " index (Re)generate every index.md in the bundle\n" + " graph Print the cross-link graph (--dot for Graphviz DOT)\n" + @@ -39,7 +41,7 @@ public static class OkfCli "OPTIONS:\n" + " -h, --help Show this help\n" + " -V, --version Show version\n" + - " --json Machine-readable output for validate/info\n" + + " --json Machine-readable output for validate/info/audit\n" + " --out Output directory for `render`"; /// @@ -86,6 +88,7 @@ public static int Run(string[] args, TextWriter stdout, TextWriter stderr) return cmd switch { "validate" => CmdValidate(rest, stdout), + "audit" => CmdAudit(rest, stdout), "info" => CmdInfo(rest, stdout), "index" => CmdIndex(rest, stdout), "graph" => CmdGraph(rest, stdout), @@ -313,6 +316,169 @@ private static int CmdValidate(string[] args, TextWriter stdout) return 1; } + /// The flags that make audit a filtered query rather than a report. + private static readonly string[] AuditFilterFlags = ["--stale", "--trust", "--status", "--type"]; + + /// Every audit flag that consumes the following token as its value. + private static readonly string[] AuditValuedFlags = ["--trust", "--status", "--type", "--as-of"]; + + /// An pinned to one date, backing --as-of. + private sealed class PinnedClock(DateOnly today) : IOkfClock + { + public DateOnly Today { get; } = today; + } + + /// Implements the audit subcommand. + private static int CmdAudit(string[] args, TextWriter stdout) + { + // Flag values are validated BEFORE the positional is resolved. An + // unvalued flag is the more specific diagnosis, and `okf audit --as-of` + // -- the flag as the only argument -- would otherwise report + // "missing " and hide the actual mistake, because Positional + // skips a valued flag's slot without checking that it has a value. + var clock = ParseAsOf(args); + + // Report mode selects exactly what --stale selects; only the + // presentation differs. --as-of and --json never switch modes. + var filtered = AuditFilterFlags.Any(f => HasFlag(args, f)); + var query = filtered ? ParseAuditQuery(args) : new AuditQuery(StaleOnly: true); + + var path = Positional(args, "", AuditValuedFlags); + var bundle = Load(path); + var report = ConceptAudit.Run(bundle, query, clock); + + if (HasFlag(args, "--json")) + { + JsonOutput.WriteAudit(stdout, path, query, report); + return 0; + } + + if (filtered) + { + foreach (var finding in report.Findings) + { + stdout.Write(FormatAuditFinding(finding)); + stdout.Write("\n"); + } + + return 0; + } + + WriteAuditReport(stdout, path, report); + return 0; + } + + /// Parses --as-of; null when absent (the audit then uses the system clock). + private static IOkfClock? ParseAsOf(string[] args) + { + var raw = FlagValue(args, "--as-of"); + if (raw is null) + { + return null; + } + + // DateOnly has no (s, format, provider, out) overload -- the five-argument + // form is the only one that takes a culture, and it is the same contract + // Lifecycle.From uses for stale_after. + if (!DateOnly.TryParseExact(raw, "yyyy-MM-dd", CultureInfo.InvariantCulture, DateTimeStyles.None, out var asOf)) + { + throw new CliOperationException($"--as-of is not a valid YYYY-MM-DD date: \"{raw}\""); + } + + return new PinnedClock(asOf); + } + + /// Builds the query from the filter flags. Throws on an unknown vocabulary value. + private static AuditQuery ParseAuditQuery(string[] args) + { + HashSet? tiers = null; + var trustRaw = FlagValue(args, "--trust"); + if (trustRaw is not null) + { + tiers = []; + foreach (var entry in trustRaw.Split(',')) + { + if (!AuditVocabulary.TryParseTrustTier(entry.Trim(), out var tier)) + { + throw new CliOperationException( + $"unknown trust tier \"{entry.Trim()}\"; expected unverified, machine-confirmed or human-reviewed"); + } + + tiers.Add(tier); + } + } + + ConceptStatus? status = null; + var statusRaw = FlagValue(args, "--status"); + if (statusRaw is not null) + { + if (!AuditVocabulary.TryParseStatus(statusRaw.Trim(), out var parsed)) + { + throw new CliOperationException( + $"unknown status \"{statusRaw.Trim()}\"; expected draft, stable or deprecated"); + } + + status = parsed; + } + + return new AuditQuery( + HasFlag(args, "--stale"), + tiers, + status, + FlagValue(args, "--type")); + } + + /// Renders one concept line: id, freshness, trust tier, status -- two spaces between fields. + private static string FormatAuditFinding(AuditFinding finding) + { + var freshness = finding.Lifecycle.StaleAfter is { } date + ? (finding.IsStale ? "stale " : "fresh ") + date.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture) + : "no-stale-after"; + + return $"{finding.Id} {freshness} {AuditVocabulary.Name(finding.Trust)} {AuditVocabulary.Name(finding.Lifecycle.Status)}"; + } + + /// Renders the report form: summary counters over the whole bundle, then the worklist. + private static void WriteAuditReport(TextWriter stdout, string bundlePath, AuditReport report) + { + stdout.Write($"bundle: {bundlePath}\n"); + stdout.Write($"as of: {report.AsOf.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture)}\n"); + stdout.Write($"concepts: {report.ConceptCount}\n"); + + // Labels always come from AuditVocabulary -- never as literals here. + // Duplicating them in each renderer is exactly the drift the shared + // vocabulary exists to prevent. Only the ORDER is decided locally: the + // report shows the strongest tier first, so it walks the canonical + // (weakest-first) list in reverse. + stdout.Write("\ntrust:\n"); + foreach (var tier in AuditVocabulary.TrustTiersInOrder.Reverse()) + { + stdout.Write($" {report.TrustCounts[tier],4} {AuditVocabulary.Name(tier)}\n"); + } + + stdout.Write("\nstatus:\n"); + foreach (var status in AuditVocabulary.StatusesInOrder) + { + stdout.Write($" {report.StatusCounts[status],4} {AuditVocabulary.Name(status)}\n"); + } + + stdout.Write($"\nstale: {report.StaleCount} of {report.ConceptCount} past stale_after\n"); + + if (report.Findings.Count == 0) + { + stdout.Write("\nneeds attention: none\n"); + return; + } + + stdout.Write($"\nneeds attention ({report.Findings.Count}):\n"); + foreach (var finding in report.Findings) + { + stdout.Write(" "); + stdout.Write(FormatAuditFinding(finding)); + stdout.Write("\n"); + } + } + /// Implements the info subcommand. private static int CmdInfo(string[] args, TextWriter stdout) { diff --git a/tests/OKF4net.Tests/CliTests.cs b/tests/OKF4net.Tests/CliTests.cs index c5c3edd..064e5d1 100644 --- a/tests/OKF4net.Tests/CliTests.cs +++ b/tests/OKF4net.Tests/CliTests.cs @@ -1,5 +1,6 @@ // SPDX-License-Identifier: LGPL-3.0-or-later using System.Linq; +using System.Text.Json; using System.Text.RegularExpressions; using OKF4net.Cli; @@ -20,6 +21,9 @@ public class CliTests // rather than assumed relative to the process's current directory. private static readonly string BundlePath = Path.Combine(TestPaths.RepoRoot(), "tests", "fixtures", "appendix_a"); + private static readonly string V02BundlePath = + Path.Combine(TestPaths.RepoRoot(), "tests", "fixtures", "okf_v02"); + private static (int Code, string Out, string Err) Run(params string[] args) => TestPaths.Run(args); [Fact] @@ -434,4 +438,228 @@ public void Render_with_only_out_and_no_bundle_fails_rather_than_treating_the_ou Assert.Contains("error:", r.Err); Assert.False(Directory.Exists(outDir)); } + + // ---------------------------------------------------------------- + // audit + // ---------------------------------------------------------------- + + [Fact] + public void Audit_report_mode_prints_summary_and_worklist() + { + var r = Run("audit", V02BundlePath, "--as-of", "2099-06-01"); + + Assert.Equal(0, r.Code); + Assert.Contains("as of: 2099-06-01\n", r.Out); + Assert.Contains("concepts: 2\n", r.Out); + Assert.Contains(" 1 human-reviewed\n", r.Out); + Assert.Contains(" 1 unverified\n", r.Out); + Assert.Contains(" 2 stable\n", r.Out); + Assert.Contains("stale: 1 of 2 past stale_after\n", r.Out); + Assert.Contains("needs attention (1):\n", r.Out); + Assert.Contains(" metrics/dau stale 2099-01-01 human-reviewed stable\n", r.Out); + } + + [Fact] + public void Audit_query_mode_prints_bare_lines_only() + { + var r = Run("audit", V02BundlePath, "--stale", "--as-of", "2099-06-01"); + + Assert.Equal(0, r.Code); + Assert.Equal("metrics/dau stale 2099-01-01 human-reviewed stable\n", r.Out); + } + + [Fact] + public void Audit_without_flags_selects_the_same_set_as_stale() + { + var report = Run("audit", V02BundlePath, "--as-of", "2099-06-01"); + var query = Run("audit", V02BundlePath, "--stale", "--as-of", "2099-06-01"); + + var reportIds = report.Out + .Split('\n') + .Where(l => l.StartsWith(" metrics/", StringComparison.Ordinal)) + .Select(l => l.Trim().Split(" ")[0]) + .ToList(); + var queryIds = query.Out + .Split('\n', StringSplitOptions.RemoveEmptyEntries) + .Select(l => l.Split(" ")[0]) + .ToList(); + + Assert.Equal(queryIds, reportIds); + } + + [Fact] + public void Audit_empty_selection_prints_nothing() + { + var r = Run("audit", V02BundlePath, "--status", "deprecated"); + + Assert.Equal(0, r.Code); + Assert.Equal("", r.Out); + } + + [Fact] + public void Audit_three_tier_idiom_returns_every_concept() + { + var r = Run("audit", V02BundlePath, "--trust", "unverified,machine-confirmed,human-reviewed"); + + Assert.Equal(0, r.Code); + Assert.Equal(2, r.Out.Split('\n', StringSplitOptions.RemoveEmptyEntries).Length); + } + + [Fact] + public void Audit_rejects_an_invalid_as_of_date() + { + var r = Run("audit", V02BundlePath, "--as-of", "2026-13-01"); + + Assert.Equal(1, r.Code); + Assert.Equal("error: --as-of is not a valid YYYY-MM-DD date: \"2026-13-01\"\n", r.Err); + } + + [Fact] + public void Audit_rejects_an_unknown_trust_tier() + { + var r = Run("audit", V02BundlePath, "--trust", "foo"); + + Assert.Equal(1, r.Code); + Assert.Equal( + "error: unknown trust tier \"foo\"; expected unverified, machine-confirmed or human-reviewed\n", + r.Err); + } + + [Fact] + public void Audit_rejects_an_unknown_status() + { + var r = Run("audit", V02BundlePath, "--status", "retired"); + + Assert.Equal(1, r.Code); + Assert.Equal("error: unknown status \"retired\"; expected draft, stable or deprecated\n", r.Err); + } + + [Fact] + public void Audit_rejects_an_empty_trust_entry_but_absorbs_duplicates() + { + var empty = Run("audit", V02BundlePath, "--trust", "unverified,,human-reviewed"); + Assert.Equal(1, empty.Code); + Assert.Contains("unknown trust tier", empty.Err); + + var duplicated = Run("audit", V02BundlePath, "--trust", "unverified,unverified"); + var single = Run("audit", V02BundlePath, "--trust", "unverified"); + Assert.Equal(0, duplicated.Code); + Assert.Equal(single.Out, duplicated.Out); + } + + /// + /// Regression guard: valued flags must be declared to Positional, or + /// their value is mistaken for the bundle path when they precede it. + /// + [Theory] + [InlineData("--as-of", "2099-06-01")] + [InlineData("--trust", "unverified")] + [InlineData("--status", "stable")] + [InlineData("--type", "Metric")] + public void Audit_valued_flags_before_the_positional_resolve_the_bundle(string flag, string value) + { + var r = Run("audit", flag, value, V02BundlePath); + + Assert.Equal(0, r.Code); + Assert.Equal("", r.Err); + } + + /// + /// A valued flag left without a value must name itself, even when it is the + /// only argument -- otherwise the user is told the bundle is missing and the + /// real mistake is hidden. This is why CmdAudit validates flag values before + /// resolving the positional. + /// + [Theory] + [InlineData("--as-of")] + [InlineData("--trust")] + [InlineData("--status")] + [InlineData("--type")] + public void Audit_reports_a_valued_flag_left_without_a_value(string flag) + { + var r = Run("audit", flag); + + Assert.Equal(1, r.Code); + Assert.Equal($"error: {flag} requires a value\n", r.Err); + } + + [Fact] + public void Audit_as_of_alone_stays_in_report_mode() + { + var r = Run("audit", V02BundlePath, "--as-of", "2099-06-01"); + + Assert.Contains("needs attention", r.Out); + } + + [Fact] + public void Help_lists_audit_right_after_validate() + { + var r = Run("--help"); + + Assert.Equal(0, r.Code); + var lines = r.Out.Split('\n').Select(l => l.TrimStart()).ToList(); + var validateIndex = lines.FindIndex(l => l.StartsWith("validate ", StringComparison.Ordinal)); + var auditIndex = lines.FindIndex(l => l.StartsWith("audit ", StringComparison.Ordinal)); + + Assert.True(validateIndex >= 0 && auditIndex == validateIndex + 1); + } + + [Fact] + public void Audit_json_carries_counts_query_and_findings() + { + var r = Run("audit", V02BundlePath, "--as-of", "2099-06-01", "--json"); + + Assert.Equal(0, r.Code); + Assert.EndsWith("\n", r.Out); + + using var doc = JsonDocument.Parse(r.Out); + var root = doc.RootElement; + + Assert.Equal("2099-06-01", root.GetProperty("asOf").GetString()); + Assert.Equal(2, root.GetProperty("conceptCount").GetInt32()); + Assert.Equal(1, root.GetProperty("staleCount").GetInt32()); + + // Report mode selects what --stale selects, so the replayed query says so. + Assert.True(root.GetProperty("query").GetProperty("stale").GetBoolean()); + Assert.Equal(JsonValueKind.Null, root.GetProperty("query").GetProperty("trust").ValueKind); + + Assert.Equal(1, root.GetProperty("trust").GetProperty("humanReviewed").GetInt32()); + Assert.Equal(1, root.GetProperty("trust").GetProperty("unverified").GetInt32()); + Assert.Equal(2, root.GetProperty("status").GetProperty("stable").GetInt32()); + + var finding = root.GetProperty("findings").EnumerateArray().Single(); + Assert.Equal("metrics/dau", finding.GetProperty("conceptId").GetString()); + Assert.Equal("Metric", finding.GetProperty("type").GetString()); + Assert.Equal("Daily Active Users", finding.GetProperty("title").GetString()); + Assert.Equal("human-reviewed", finding.GetProperty("trust").GetString()); + Assert.Equal("2099-01-01", finding.GetProperty("staleAfter").GetString()); + Assert.True(finding.GetProperty("stale").GetBoolean()); + } + + [Fact] + public void Audit_json_serializes_trust_query_in_ladder_order() + { + var r = Run("audit", V02BundlePath, "--trust", "human-reviewed,unverified", "--json"); + + using var doc = JsonDocument.Parse(r.Out); + var trust = doc.RootElement.GetProperty("query").GetProperty("trust") + .EnumerateArray().Select(e => e.GetString()).ToList(); + + Assert.Equal(["unverified", "human-reviewed"], trust); + } + + [Fact] + public void Audit_json_keeps_a_malformed_stale_after_raw_and_not_stale() + { + using var tmp = new TempDir(); + tmp.Write("a.md", "---\ntype: Metric\nstale_after: not-a-date\n---\n"); + + var r = Run("audit", tmp.Path, "--trust", "unverified", "--json"); + + using var doc = JsonDocument.Parse(r.Out); + var finding = doc.RootElement.GetProperty("findings").EnumerateArray().Single(); + + Assert.Equal("not-a-date", finding.GetProperty("staleAfter").GetString()); + Assert.False(finding.GetProperty("stale").GetBoolean()); + } } From 6c74d47aeb14d26b0b4a9da75a84fc69e243dfa7 Mon Sep 17 00:00:00 2001 From: Julien CHABLE Date: Fri, 21 Aug 2026 23:26:15 +0200 Subject: [PATCH 10/19] test(audit): pin okf audit's text and JSON output with goldens Hand-authored against the v0.2 spec (audit has no upstream reference binary), verified by running okf4net's own audit against tests/fixtures/okf_v02 before saving. Both goldens matched the predicted text exactly. The JSON parity test's Windows-only backslash normalization needed a fix along the way: r.Out is serialized JSON, where a native backslash separator is escaped to the two-character sequence `\` in the JSON text itself. The originally drafted `r.Out.Replace('\', '/')` replaced each of those two characters individually, turning one path separator into "//" instead of "/". Matching on the two-character escaped sequence instead of the bare char fixes it. --- tests/OKF4net.Tests/GoldenParityTests.cs | 39 ++++++++++++++++++++++++ tests/fixtures/README.md | 9 ++++++ tests/fixtures/golden/audit-v02.json | 1 + tests/fixtures/golden/audit-v02.out | 18 +++++++++++ 4 files changed, 67 insertions(+) create mode 100644 tests/fixtures/golden/audit-v02.json create mode 100644 tests/fixtures/golden/audit-v02.out diff --git a/tests/OKF4net.Tests/GoldenParityTests.cs b/tests/OKF4net.Tests/GoldenParityTests.cs index 3fce43c..f5dd381 100644 --- a/tests/OKF4net.Tests/GoldenParityTests.cs +++ b/tests/OKF4net.Tests/GoldenParityTests.cs @@ -117,6 +117,45 @@ public void Info_output_matches_golden() Assert.Equal(Golden("info.out"), r.Out); } + /// + /// audit's goldens are hand-authored and verified against the spec + /// text (§5.3 tiers, §5.4 statuses, §5.5 staleness), not captured from the + /// reference CLI -- the verb has no upstream counterpart. The date is + /// pinned with --as-of so the output cannot drift with the calendar. + /// There is no audit-v02.exitcode: the verb always exits 0, so the + /// code is asserted inline (as does). + /// + [Fact] + public void Audit_report_matches_golden() + { + var r = WithRepoRootAsCwd(() => Run("audit", "tests/fixtures/okf_v02", "--as-of", "2099-06-01")); + + Assert.Equal(0, r.Code); + + // Every path in this output is a concept id, always '/'-normalized by + // ConceptId.FromPath, so the comparison is strict byte-for-byte. + Assert.Equal(Golden("audit-v02.out"), r.Out); + } + + [Fact] + public void Audit_json_matches_golden() + { + var r = WithRepoRootAsCwd(() => Run("audit", "tests/fixtures/okf_v02", "--as-of", "2099-06-01", "--json")); + + Assert.Equal(0, r.Code); + + // Only the findings' `path` field carries a native separator (it is a + // real file path, not a concept id), so it alone is normalized in the + // C# OUTPUT -- never in the golden -- exactly like validate.out. Unlike + // validate.out (plain text, one real backslash char per separator), + // r.Out here is serialized JSON: JsonSerializer escapes each backslash + // as the two-character sequence `\\` in the JSON text itself. A naive + // single-char Replace('\\', '/') would turn that pair into "//" instead + // of collapsing it to one '/', so the search pattern below is the + // two-character escaped sequence, not the bare character. + Assert.Equal(Golden("audit-v02.json"), r.Out.Replace("\\\\", "/")); + } + [Fact] public void Graph_dot_matches_golden() { diff --git a/tests/fixtures/README.md b/tests/fixtures/README.md index 50b7517..2591ba7 100644 --- a/tests/fixtures/README.md +++ b/tests/fixtures/README.md @@ -192,3 +192,12 @@ them a re-capture from the (removed) Rust binary: (same diagnostic construction, different caught exception type), so the risk of it being wrong is low, but it remains unexercised by an automated test. + +## `okf audit` goldens (2026-08-21) + +- `golden/audit-v02.out`, `golden/audit-v02.json` — output of + `okf audit tests/fixtures/okf_v02 --as-of 2099-06-01` (and its `--json` + form). **Hand-authored**, verified against the spec text (§5.3 trust tiers, + §5.4 statuses, §5.5 staleness) rather than captured from the reference CLI: + `audit` is an OKF4net verb with no upstream counterpart. The `--as-of` date + is pinned so the output cannot drift with the calendar. diff --git a/tests/fixtures/golden/audit-v02.json b/tests/fixtures/golden/audit-v02.json new file mode 100644 index 0000000..23303ad --- /dev/null +++ b/tests/fixtures/golden/audit-v02.json @@ -0,0 +1 @@ +{"bundle":"tests/fixtures/okf_v02","asOf":"2099-06-01","conceptCount":2,"query":{"stale":true,"trust":null,"status":null,"type":null},"trust":{"humanReviewed":1,"machineConfirmed":0,"unverified":1},"status":{"draft":0,"stable":2,"deprecated":0},"staleCount":1,"findings":[{"conceptId":"metrics/dau","path":"tests/fixtures/okf_v02/metrics/dau.md","type":"Metric","title":"Daily Active Users","trust":"human-reviewed","status":"stable","staleAfter":"2099-01-01","stale":true}]} diff --git a/tests/fixtures/golden/audit-v02.out b/tests/fixtures/golden/audit-v02.out new file mode 100644 index 0000000..a0a7486 --- /dev/null +++ b/tests/fixtures/golden/audit-v02.out @@ -0,0 +1,18 @@ +bundle: tests/fixtures/okf_v02 +as of: 2099-06-01 +concepts: 2 + +trust: + 1 human-reviewed + 0 machine-confirmed + 1 unverified + +status: + 0 draft + 2 stable + 0 deprecated + +stale: 1 of 2 past stale_after + +needs attention (1): + metrics/dau stale 2099-01-01 human-reviewed stable From 5d42e66fe50f99e3e275c64f78376f7a37ce746b Mon Sep 17 00:00:00 2001 From: Julien CHABLE Date: Fri, 21 Aug 2026 23:26:26 +0200 Subject: [PATCH 11/19] test(cli): cover audit's empty-worklist and no-stale-after cases Two CLI-level review gaps closed: - Audit_report_mode_prints_none_when_nothing_is_stale: report mode's "needs attention: none" branch (WriteAuditReport's empty-Findings early return) had no test exercising it. - Audit_finding_line_reports_no_stale_after_when_the_field_is_absent: FormatAuditFinding's "no-stale-after" freshness label (emitted when a concept has no stale_after at all, as metrics/legacy does) was never asserted. Also corrects JsonOutput's class doc comment, stale since WriteAudit was added: it still credited only validate/info. --- src/OKF4net.Cli/JsonOutput.cs | 2 +- tests/OKF4net.Tests/CliTests.cs | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+), 1 deletion(-) diff --git a/src/OKF4net.Cli/JsonOutput.cs b/src/OKF4net.Cli/JsonOutput.cs index aef0bef..9b8ad62 100644 --- a/src/OKF4net.Cli/JsonOutput.cs +++ b/src/OKF4net.Cli/JsonOutput.cs @@ -86,7 +86,7 @@ internal partial class CliJsonContext : JsonSerializerContext { } -/// Builds and writes the --json output for validate and info. +/// Builds and writes the --json output for validate, info and audit. internal static class JsonOutput { /// Writes okf validate --json's result to as a single line-terminated JSON document. diff --git a/tests/OKF4net.Tests/CliTests.cs b/tests/OKF4net.Tests/CliTests.cs index 064e5d1..cb5382f 100644 --- a/tests/OKF4net.Tests/CliTests.cs +++ b/tests/OKF4net.Tests/CliTests.cs @@ -591,6 +591,39 @@ public void Audit_as_of_alone_stays_in_report_mode() Assert.Contains("needs attention", r.Out); } + /// + /// Report mode's empty-worklist branch: --as-of pinned before + /// metrics/dau's stale_after (2099-01-01) leaves nothing + /// stale, so WriteAuditReport takes its early-return branch and + /// prints the "none" line instead of a "needs attention (N):" worklist. + /// + [Fact] + public void Audit_report_mode_prints_none_when_nothing_is_stale() + { + var r = Run("audit", V02BundlePath, "--as-of", "2026-01-01"); + + Assert.Equal(0, r.Code); + Assert.Contains("stale: 0 of 2 past stale_after\n", r.Out); + Assert.Contains("needs attention: none\n", r.Out); + Assert.DoesNotContain("needs attention (", r.Out); + } + + /// + /// metrics/legacy has no stale_after at all, so + /// FormatAuditFinding takes its "no-stale-after" branch rather than + /// "stale "/"fresh " + a date. --trust unverified selects exactly + /// this concept (it has no verified entries), independent of + /// --as-of/the system clock. + /// + [Fact] + public void Audit_finding_line_reports_no_stale_after_when_the_field_is_absent() + { + var r = Run("audit", V02BundlePath, "--trust", "unverified"); + + Assert.Equal(0, r.Code); + Assert.Equal("metrics/legacy no-stale-after unverified stable\n", r.Out); + } + [Fact] public void Help_lists_audit_right_after_validate() { From 6dd9e8d46bfcd96707b0a68e13f35d371a0730a8 Mon Sep 17 00:00:00 2001 From: Julien CHABLE Date: Fri, 21 Aug 2026 23:38:13 +0200 Subject: [PATCH 12/19] feat(agents): expose okf_audit as a read-only tool Adds a PinnedClock IOkfClock adapter over the existing UtcNow/Today seam and a RenderAudit helper (deliberately separate from the CLI's golden-locked renderer) so okf_audit surfaces ConceptAudit's trust/freshness/lifecycle signals to agents, capped at 20 findings and never throwing (RunTool guard). Not added to WriteToolNames: it is read-only. Updates the existing tests that hard-code OkfBundleTools' tool count/order and the MCP server's read-only-mode tool count now that an 11th unconditional tool exists, and syncs the two READMEs' tool tables/counts accordingly. --- README.md | 7 +- src/OKF4net.Agents/OkfBundleTools.cs | 138 +++++++++++++++++- src/OKF4net.Mcp/README.md | 23 +-- .../Agents/AIFunctionExposureTests.cs | 22 +-- .../OKF4net.Tests/Agents/OkfAuditToolTests.cs | 121 +++++++++++++++ .../Agents/OkfBundleToolsTests.cs | 3 +- tests/OKF4net.Tests/Mcp/OkfMcpServerTests.cs | 16 +- 7 files changed, 297 insertions(+), 33 deletions(-) create mode 100644 tests/OKF4net.Tests/Agents/OkfAuditToolTests.cs diff --git a/README.md b/README.md index 870a0cd..49ed468 100644 --- a/README.md +++ b/README.md @@ -231,9 +231,9 @@ var response = await agent.RunAsync("Search the bundle for concepts about refund Console.WriteLine(response.Text); ``` -The ten unconditional tools, plus the eleventh conditional on an attestation -orchestrator being wired (read → browse → graph → search → write → append → -regenerate → validate → changes-since → get-computation → run-computation): +The eleven unconditional tools, plus the twelfth conditional on an attestation +orchestrator being wired (read → browse → graph → search → audit → write → +append → regenerate → validate → changes-since → get-computation → run-computation): | Tool | Description | |--------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| @@ -241,6 +241,7 @@ regenerate → validate → changes-since → get-computation → run-computatio | `okf_browse` | Browse the bundle via its index files (progressive disclosure). Without a path, lists the bundle root. | | `okf_graph` | Inspect the cross-link graph. With a concept id: its outgoing links, backlinks and broken links. Without: bundle-wide stats. | | `okf_search` | Full-text search across concept titles, descriptions, tags and bodies. Returns matching concept ids ranked by relevance. | +| `okf_audit` | Audit the bundle's trust, freshness and lifecycle signals (§5.3–§5.5): counts by trust tier and status, plus the concepts needing attention. Read-only. | | `okf_write_concept` | Create or update a concept document. The frontmatter must contain non-empty type, title and description (producer-grade validation is enforced before writing). | | `okf_append_log` | Append an entry to the bundle root log.md under today's date (ISO). Note: log.md is re-rendered through the strict §9 model, so non-conforming prose or comments in a hand-authored log.md are not preserved. | | `okf_regenerate_indexes` | Regenerate every index.md in the bundle (progressive-disclosure listings). Run after adding or changing concepts. | diff --git a/src/OKF4net.Agents/OkfBundleTools.cs b/src/OKF4net.Agents/OkfBundleTools.cs index 0a3b556..db4ac8a 100644 --- a/src/OKF4net.Agents/OkfBundleTools.cs +++ b/src/OKF4net.Agents/OkfBundleTools.cs @@ -25,6 +25,11 @@ public sealed class OkfBundleTools "Usage: okf_changes_since requires a valid ISO date (yyyy-MM-dd), inclusive. " + "Example: okf_changes_since(\"2026-01-01\")."; + private const string AuditUsageMessage = + "Usage: okf_audit takes optional filters — stale (bool), trust (comma-separated: " + + "unverified, machine-confirmed, human-reviewed), status (draft, stable or deprecated) " + + "and type (exact frontmatter type). Example: okf_audit(stale: true, trust: \"unverified\")."; + /// /// The core write primitive this tool set delegates every write to: /// producer-validated create/update () and @@ -138,6 +143,16 @@ public OkfBundleTools(string bundleRoot, AttestationOrchestrator? orchestrator) /// Today's date, derived from — the shared seam behind 's and 's staleness checks. private DateOnly Today => DateOnly.FromDateTime(UtcNow().Date); + /// + /// Pins to — the same + /// UtcNow seam and use — so + /// the tool's output never depends on the day it runs. + /// + private sealed class PinnedClock(DateOnly today) : IOkfClock + { + public DateOnly Today { get; } = today; + } + /// /// Returns the loaded bundle, loading it from on /// first access and caching it thereafter until @@ -194,7 +209,7 @@ internal void InvalidateBundle() /// from each method's own /// — the single source of truth, so /// the two can never drift apart. The order is stable: read → browse → - /// graph → search → write → append → regenerate → validate → + /// graph → search → audit → write → append → regenerate → validate → /// changes-since → get-computation → (conditionally) run-computation. /// /// okf_get_computation is always included — it is read-only and @@ -213,6 +228,7 @@ public IList GetTools() AIFunctionFactory.Create(Browse, "okf_browse"), AIFunctionFactory.Create(Graph, "okf_graph"), AIFunctionFactory.Create(Search, "okf_search"), + AIFunctionFactory.Create(Audit, "okf_audit"), AIFunctionFactory.Create(WriteConcept, "okf_write_concept"), AIFunctionFactory.Create(AppendLog, "okf_append_log"), AIFunctionFactory.Create(RegenerateIndexes, "okf_regenerate_indexes"), @@ -446,6 +462,66 @@ public string Search( .Select(s => (s.Concept, s.Score)) .ToList(); + /// + /// Audits the bundle's trust, freshness and lifecycle signals (§5.3–§5.5): + /// counts over the whole bundle, then the concepts the filters select, + /// bounded to 20 entries. + /// + /// Keep only concepts past their stale_after date. + /// Comma-separated trust tiers to keep. + /// Keep only concepts with this lifecycle status. + /// Keep only concepts with this frontmatter type (exact match). + [Description("Audit the bundle's trust, freshness and lifecycle signals: counts by trust tier and status, plus the concepts needing attention. Filter with stale/trust/status/type.")] + public string Audit( + [Description("Only concepts past their stale_after date. Defaults to true.")] bool stale = true, + [Description("Comma-separated trust tiers to include: unverified, machine-confirmed, human-reviewed.")] string? trust = null, + [Description("Only concepts with this lifecycle status: draft, stable or deprecated.")] string? status = null, + [Description("Only concepts with this frontmatter type (exact match).")] string? type = null) + { + HashSet? tiers = null; + if (trust is not null) + { + tiers = []; + foreach (var entry in trust.Split(',')) + { + if (!AuditVocabulary.TryParseTrustTier(entry.Trim(), out var tier)) + { + return AuditUsageMessage; + } + + tiers.Add(tier); + } + } + + ConceptStatus? parsedStatus = null; + if (status is not null) + { + if (!AuditVocabulary.TryParseStatus(status.Trim(), out var value)) + { + return AuditUsageMessage; + } + + parsedStatus = value; + } + + // Everything that can touch the filesystem goes through RunTool, the + // guard every bundle-loading tool uses: it turns OkfException (hence + // BundleLoadException), ArgumentException, IOException, + // UnauthorizedAccessException and DecoderFallbackException into an + // "Error: ..." string. A function tool must return an error, not throw + // one -- a directory deleted after construction would otherwise escape + // as an exception into the agent runtime. + return RunTool(() => + { + var report = ConceptAudit.Run( + GetBundle(), + new AuditQuery(stale, tiers, parsedStatus, type), + new PinnedClock(Today)); + + return RenderAudit(report); + }); + } + /// /// Creates or updates one concept document. Producer-grade validation /// (: non-empty type, @@ -1083,6 +1159,66 @@ private static string FormatSearchResults(string query, string? tag, IReadOnlyLi return sb.ToString(); } + /// + /// Renders an audit for an agent: the same shape as the CLI's report form, + /// minus the bundle line (the tool is bound to one bundle) and bounded to + /// 20 findings. Deliberately not shared with the CLI renderer, whose bytes + /// are golden-locked and must not move when this string is tuned. + /// + private static string RenderAudit(AuditReport report) + { + const int MaxResults = 20; + var sb = new StringBuilder(); + + sb.Append("as of: ").Append(report.AsOf.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture)).Append('\n'); + sb.Append("concepts: ").Append(report.ConceptCount).Append('\n'); + + // Same rule as the CLI renderer: labels from AuditVocabulary, never + // literals. The two renderers are separate on purpose (the CLI's bytes + // are golden-locked), but they must not spell the vocabulary twice. + sb.Append("\ntrust:\n"); + foreach (var tier in AuditVocabulary.TrustTiersInOrder.Reverse()) + { + sb.Append($" {report.TrustCounts[tier],4} {AuditVocabulary.Name(tier)}\n"); + } + + sb.Append("\nstatus:\n"); + foreach (var status in AuditVocabulary.StatusesInOrder) + { + sb.Append($" {report.StatusCounts[status],4} {AuditVocabulary.Name(status)}\n"); + } + + sb.Append($"\nstale: {report.StaleCount} of {report.ConceptCount} past stale_after\n"); + + if (report.Findings.Count == 0) + { + sb.Append("\nneeds attention: none\n"); + return sb.ToString(); + } + + sb.Append($"\nneeds attention ({report.Findings.Count}):\n"); + foreach (var finding in report.Findings.Take(MaxResults)) + { + var freshness = finding.Lifecycle.StaleAfter is { } date + ? (finding.IsStale ? "stale " : "fresh ") + date.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture) + : "no-stale-after"; + + sb.Append(" ") + .Append(finding.Id) + .Append(" ").Append(freshness) + .Append(" ").Append(AuditVocabulary.Name(finding.Trust)) + .Append(" ").Append(AuditVocabulary.Name(finding.Lifecycle.Status)) + .Append('\n'); + } + + if (report.Findings.Count > MaxResults) + { + sb.Append($"… and {report.Findings.Count - MaxResults} more (narrow with stale/trust/status/type)\n"); + } + + return sb.ToString(); + } + /// /// Builds the fallback listing for a bundle level with no index.md: /// the subdirectories and concepts found directly under diff --git a/src/OKF4net.Mcp/README.md b/src/OKF4net.Mcp/README.md index f42b528..bbc7e6a 100644 --- a/src/OKF4net.Mcp/README.md +++ b/src/OKF4net.Mcp/README.md @@ -68,20 +68,21 @@ or `OKF_BUNDLE_ROOT` in `claude_desktop_config.json`. ## Tools -`okf_read_concept`, `okf_browse`, `okf_graph`, `okf_search`, `okf_write_concept`, -`okf_append_log`, `okf_regenerate_indexes`, `okf_validate_bundle`, +`okf_read_concept`, `okf_browse`, `okf_graph`, `okf_search`, `okf_audit`, +`okf_write_concept`, `okf_append_log`, `okf_regenerate_indexes`, `okf_validate_bundle`, `okf_changes_since`, `okf_get_computation`. Each is the corresponding `OkfBundleTools` operation, so all OKF v0.2 behaviour, producer-grade validation, path-safety, and locking apply unchanged. -That's ten tools full (seven read-only tools above plus the three write -tools), or seven when `OKF_MCP_READONLY=1` drops the three write tools. +That's eleven tools full (eight read-only tools above plus the three write +tools), or eight when `OKF_MCP_READONLY=1` drops the three write tools. `okf_get_computation` reads a §10 attested-computation concept's contract and -sanctioned computation source — read-only, no attestation runtime needed. The -eleventh `OkfBundleTools` tool, `okf_run_computation`, is **not** exposed by -this server: it only appears in `GetTools()` when the tool set is constructed -with an `OKF4net.Attestation` `AttestationOrchestrator` wired in, and this -server starts `OkfBundleTools` with no orchestrator (it wires no -host-specific binder/executor/attester runtime). Embed `OKF4net.Agents` -directly if you need `okf_run_computation`. +sanctioned computation source — read-only, no attestation runtime needed. +`okf_audit` reads the bundle's trust/freshness/lifecycle signals — also +read-only. The twelfth `OkfBundleTools` tool, `okf_run_computation`, is +**not** exposed by this server: it only appears in `GetTools()` when the tool +set is constructed with an `OKF4net.Attestation` `AttestationOrchestrator` +wired in, and this server starts `OkfBundleTools` with no orchestrator (it +wires no host-specific binder/executor/attester runtime). Embed +`OKF4net.Agents` directly if you need `okf_run_computation`. diff --git a/tests/OKF4net.Tests/Agents/AIFunctionExposureTests.cs b/tests/OKF4net.Tests/Agents/AIFunctionExposureTests.cs index de95130..4b81f0b 100644 --- a/tests/OKF4net.Tests/Agents/AIFunctionExposureTests.cs +++ b/tests/OKF4net.Tests/Agents/AIFunctionExposureTests.cs @@ -9,13 +9,14 @@ namespace OKF4net.Tests.Agents; /// -/// Tests : the ten tool methods exposed -/// as Agent Framework s (via ) -/// when no attestation orchestrator is wired (so okf_run_computation -/// is omitted; see for the wired -/// case), with no LLM involved — everything is verified at the -/// level, including a real end-to-end invocation -/// that proves argument binding from a plain dictionary works. +/// Tests : the eleven tool methods +/// exposed as Agent Framework s (via +/// ) when no attestation orchestrator is wired (so +/// okf_run_computation is omitted; see +/// for the wired case), with no LLM +/// involved — everything is verified at the level, +/// including a real end-to-end invocation that proves argument binding from +/// a plain dictionary works. /// public class AIFunctionExposureTests { @@ -27,6 +28,7 @@ public class AIFunctionExposureTests "okf_browse", "okf_graph", "okf_search", + "okf_audit", "okf_write_concept", "okf_append_log", "okf_regenerate_indexes", @@ -36,14 +38,14 @@ public class AIFunctionExposureTests ]; [Fact] - public void GetTools_returns_exactly_ten_tools() + public void GetTools_returns_exactly_eleven_tools() { var tools = new OkfBundleTools(BundlePath); - Assert.Equal(10, tools.GetTools().Count); + Assert.Equal(11, tools.GetTools().Count); } [Fact] - public void GetTools_names_are_the_ten_snake_case_names_in_stable_order() + public void GetTools_names_are_the_eleven_snake_case_names_in_stable_order() { var tools = new OkfBundleTools(BundlePath); var names = tools.GetTools().Cast().Select(f => f.Name).ToList(); diff --git a/tests/OKF4net.Tests/Agents/OkfAuditToolTests.cs b/tests/OKF4net.Tests/Agents/OkfAuditToolTests.cs new file mode 100644 index 0000000..77639cb --- /dev/null +++ b/tests/OKF4net.Tests/Agents/OkfAuditToolTests.cs @@ -0,0 +1,121 @@ +// SPDX-License-Identifier: LGPL-3.0-or-later +using Microsoft.Extensions.AI; +using OKF4net.Agents; + +namespace OKF4net.Tests.Agents; + +/// +/// Tests for the okf_audit tool. Every test pins UtcNow: the tool +/// deliberately exposes no asOf parameter, so the shared clock seam is +/// the only way its output can be made deterministic. +/// +public class OkfAuditToolTests +{ + private static OkfBundleTools ToolsOver(TempDir tmp, DateOnly today) + => new(tmp.Path) + { + UtcNow = () => new DateTime(today.Year, today.Month, today.Day, 0, 0, 0, DateTimeKind.Utc), + }; + + [Fact] + public void Audit_is_registered_and_read_only() + { + var tools = new OkfBundleTools(Path.Combine(TestPaths.RepoRoot(), "tests", "fixtures", "okf_v02")); + + Assert.Contains("okf_audit", tools.GetTools().OfType().Select(t => t.Name)); + Assert.DoesNotContain("okf_audit", OkfBundleTools.WriteToolNames); + } + + /// + /// §5.5's boundary is today >= stale_after, so a concept whose + /// stale_after is exactly today is stale. Without the pinned seam this + /// assertion would silently depend on the day the suite runs. + /// + [Fact] + public void Audit_treats_today_equals_stale_after_as_stale() + { + using var tmp = new TempDir(); + tmp.Write("a.md", "---\ntype: Metric\nstale_after: 2026-08-21\n---\n"); + + var onTheDay = ToolsOver(tmp, new DateOnly(2026, 8, 21)).Audit(); + var theDayBefore = ToolsOver(tmp, new DateOnly(2026, 8, 20)).Audit(); + + Assert.Contains("a stale 2026-08-21", onTheDay); + Assert.Contains("needs attention: none", theDayBefore); + } + + [Fact] + public void Audit_reports_counts_and_omits_the_bundle_line() + { + using var tmp = new TempDir(); + tmp.Write("a.md", "---\ntype: Metric\nverified:\n - { by: human:ada, at: 2026-01-01T00:00:00Z }\n---\n"); + tmp.Write("b.md", "---\ntype: Metric\n---\n"); + + var text = ToolsOver(tmp, new DateOnly(2026, 8, 21)).Audit(); + + Assert.DoesNotContain("bundle:", text); + Assert.Contains("as of: 2026-08-21", text); + Assert.Contains(" 1 human-reviewed", text); + Assert.Contains(" 1 unverified", text); + } + + [Fact] + public void Audit_caps_the_listing_at_twenty_findings() + { + using var tmp = new TempDir(); + for (var i = 0; i < 25; i++) + { + tmp.Write($"c{i:D2}.md", "---\ntype: Metric\nstale_after: 2026-01-01\n---\n"); + } + + var text = ToolsOver(tmp, new DateOnly(2026, 8, 21)).Audit(); + + Assert.Contains("… and 5 more (narrow with stale/trust/status/type)", text); + + // Finding lines are the two-space-indented ones; matching on "c" alone + // would also catch the "concepts:" header. + Assert.Equal(20, text.Split('\n').Count(l => l.StartsWith(" c", StringComparison.Ordinal))); + } + + [Fact] + public void Audit_renders_a_usage_message_for_invalid_vocabulary() + { + using var tmp = new TempDir(); + tmp.Write("a.md", "---\ntype: Metric\n---\n"); + + var tools = ToolsOver(tmp, new DateOnly(2026, 8, 21)); + + Assert.Contains("Usage: okf_audit", tools.Audit(trust: "machine")); + Assert.Contains("Usage: okf_audit", tools.Audit(status: "retired")); + } + + /// + /// A function tool returns errors, it does not throw them: a bundle that + /// disappears after the tool was constructed must surface as an "Error: ..." + /// string, which is what the shared RunTool guard provides. + /// + [Fact] + public void Audit_returns_an_error_string_when_the_bundle_cannot_be_loaded() + { + var tmp = new TempDir(); + tmp.Write("a.md", "---\ntype: Metric\n---\n"); + var tools = ToolsOver(tmp, new DateOnly(2026, 8, 21)); + tmp.Dispose(); // the directory is gone before the first load + + var text = tools.Audit(); + + Assert.StartsWith("Error: ", text); + } + + [Fact] + public void Audit_with_stale_false_and_no_filter_returns_the_whole_corpus() + { + using var tmp = new TempDir(); + tmp.Write("a.md", "---\ntype: Metric\n---\n"); + tmp.Write("b.md", "---\ntype: Metric\n---\n"); + + var text = ToolsOver(tmp, new DateOnly(2026, 8, 21)).Audit(stale: false); + + Assert.Contains("needs attention (2):", text); + } +} diff --git a/tests/OKF4net.Tests/Agents/OkfBundleToolsTests.cs b/tests/OKF4net.Tests/Agents/OkfBundleToolsTests.cs index 154f03f..6fd5fb5 100644 --- a/tests/OKF4net.Tests/Agents/OkfBundleToolsTests.cs +++ b/tests/OKF4net.Tests/Agents/OkfBundleToolsTests.cs @@ -53,12 +53,13 @@ public void WriteToolNames_matches_the_three_mutating_tools_and_filters_them_out .Where(name => !OkfBundleTools.WriteToolNames.Contains(name)) .ToHashSet(); - Assert.Equal(7, readOnlyNames.Count); + Assert.Equal(8, readOnlyNames.Count); Assert.DoesNotContain("okf_write_concept", readOnlyNames); Assert.DoesNotContain("okf_append_log", readOnlyNames); Assert.DoesNotContain("okf_regenerate_indexes", readOnlyNames); Assert.Contains("okf_read_concept", readOnlyNames); Assert.Contains("okf_get_computation", readOnlyNames); + Assert.Contains("okf_audit", readOnlyNames); } /// diff --git a/tests/OKF4net.Tests/Mcp/OkfMcpServerTests.cs b/tests/OKF4net.Tests/Mcp/OkfMcpServerTests.cs index 2ea625a..1c8de63 100644 --- a/tests/OKF4net.Tests/Mcp/OkfMcpServerTests.cs +++ b/tests/OKF4net.Tests/Mcp/OkfMcpServerTests.cs @@ -76,7 +76,7 @@ public async Task Write_then_read_round_trips_through_mcp() } [Fact] - public async Task Build_exposes_all_ten_tools() + public async Task Build_exposes_all_eleven_tools() { var bundle = NewBundleDir(); try @@ -92,7 +92,7 @@ public async Task Build_exposes_all_ten_tools() Assert.Equal( new[] { - "okf_append_log", "okf_browse", "okf_changes_since", "okf_get_computation", + "okf_append_log", "okf_audit", "okf_browse", "okf_changes_since", "okf_get_computation", "okf_graph", "okf_read_concept", "okf_regenerate_indexes", "okf_search", "okf_validate_bundle", "okf_write_concept", }, @@ -122,7 +122,7 @@ public async Task Build_readOnly_omits_the_three_write_tools() var names = (await client.ListToolsAsync()).Select(t => t.Name).ToHashSet(); - Assert.Equal(7, names.Count); + Assert.Equal(8, names.Count); Assert.DoesNotContain("okf_write_concept", names); Assert.DoesNotContain("okf_append_log", names); Assert.DoesNotContain("okf_regenerate_indexes", names); @@ -130,6 +130,8 @@ public async Task Build_readOnly_omits_the_three_write_tools() // okf_get_computation is read-only and needs no attestation runtime, // so it surfaces in read-only mode too -- this is deliberate. Assert.Contains("okf_get_computation", names); + // okf_audit is read-only too, so it surfaces in read-only mode. + Assert.Contains("okf_audit", names); } finally { @@ -138,7 +140,7 @@ public async Task Build_readOnly_omits_the_three_write_tools() } [Fact] - public void ConfigureServices_registers_all_ten_tools() + public void ConfigureServices_registers_all_eleven_tools() { var bundle = NewBundleDir(); try @@ -147,7 +149,7 @@ public void ConfigureServices_registers_all_ten_tools() OkfMcpHost.ConfigureServices(services, bundle, readOnly: false, version: "0.0.0"); using var provider = services.BuildServiceProvider(); var options = provider.GetRequiredService>().Value; - Assert.Equal(10, options.ToolCollection?.Count); + Assert.Equal(11, options.ToolCollection?.Count); } finally { @@ -156,7 +158,7 @@ public void ConfigureServices_registers_all_ten_tools() } [Fact] - public void ConfigureServices_readOnly_registers_seven_tools() + public void ConfigureServices_readOnly_registers_eight_tools() { var bundle = NewBundleDir(); try @@ -165,7 +167,7 @@ public void ConfigureServices_readOnly_registers_seven_tools() OkfMcpHost.ConfigureServices(services, bundle, readOnly: true, version: "0.0.0"); using var provider = services.BuildServiceProvider(); var options = provider.GetRequiredService>().Value; - Assert.Equal(7, options.ToolCollection?.Count); + Assert.Equal(8, options.ToolCollection?.Count); } finally { From bea824f3a442527b4de4d3adbc1fcbd155a8f245 Mon Sep 17 00:00:00 2001 From: Julien CHABLE Date: Fri, 21 Aug 2026 23:45:42 +0200 Subject: [PATCH 13/19] docs(audit): document the okf audit verb and okf_audit tool Co-Authored-By: Claude Sonnet 5 --- CHANGELOG.md | 6 ++++++ README.md | 17 +++++++++++++++++ 2 files changed, 23 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 800e89c..2e4db9e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,12 @@ and this project adheres to ### Added +- `okf audit` — a corpus-level query over a bundle's trust (§5.3), lifecycle + (§5.4) and staleness (§5.5) signals: counts plus a filterable worklist, with + `--stale`, `--trust`, `--status`, `--type`, `--as-of` and `--json`. Backed by + the new `ConceptAudit` in the core library and exposed to agents as the + read-only `okf_audit` tool. + - **`okf render --out `** generates a self-contained, browsable HTML site from a bundle: one page per concept (frontmatter table + rendered body), a generated index, navigable cross-links with broken links flagged, diff --git a/README.md b/README.md index 49ed468..9e72896 100644 --- a/README.md +++ b/README.md @@ -176,6 +176,7 @@ On any OS, build from source — see [Building & testing](#building--testing). ``` okf validate Check a bundle against OKF v0.2 conformance (§11) +okf audit Query trust, freshness and lifecycle signals (§5.3–§5.5) okf info Summarize a bundle (concepts, types, links, version) okf index (Re)generate every index.md in the bundle okf graph Print the cross-link graph (--dot for Graphviz DOT) @@ -192,6 +193,21 @@ okf validate ./bundles/ga4 okf graph ./bundles/ga4 --dot | dot -Tsvg > graph.svg ``` +`okf audit ` reports trust, freshness and lifecycle across a whole +bundle: counts per trust tier (§5.3) and status (§5.4), plus the worklist of +stale concepts (§5.5). Filter it to ask corpus-level questions — + +```sh +# Which concepts are past stale_after and were never verified by a human? +okf audit bundles/acme_retail --stale --trust unverified,machine-confirmed +``` + +Without filter flags it selects exactly what `--stale` selects and prints the +summary form; with any filter flag it prints one line per matching concept, so +the output pipes. `--json` always emits the full document. Note the counts +always cover the whole bundle while `findings` covers the selection: `audit` is +a worklist, not an inventory (use `okf info --json` for that). + Generate a browsable HTML site from a bundle: ```sh @@ -508,6 +524,7 @@ This table is also published as the | §4 Concept documents | `OKF4net.OkfDocument`, `OKF4net.Frontmatter` | | §4.2 Body headings | `OkfDocument.Computation()` (fenced `# Computation` heading) | | §5 Provenance, trust, and lifecycle | `Frontmatter.Sources`/`Generated`/`Verified`/`TrustTier`/`Status`/`StaleAfter`, `Actor`/`Trust`/`Provenance`/`Lifecycle` | +| §5.3–§5.5 | `ConceptAudit`, `AuditQuery`, `AuditReport` — corpus-level trust/freshness query behind `okf audit` and `okf_audit` | | §6 Cross-linking and paths | `OKF4net.LinkScanner`, `Bundle.LinksFrom` / `Bundle.Backlinks` | | §6.2 Path-valued fields | `OkfDocument.FrontmatterResources()`, `Bundle.TryResolveResource` / `Bundle.ReadResourceText` | | §7 Actor convention | `OKF4net.Actor.Parse` — `human:`/`process:`/`/` | From 6bd7ab64cca8c1798b3be91184914931484c02cb Mon Sep 17 00:00:00 2001 From: Julien CHABLE Date: Sat, 22 Aug 2026 07:15:58 +0200 Subject: [PATCH 14/19] fix(audit): stop okf_audit mislabeling fresh selections, share vocabulary spellings - RenderAudit now takes staleOnly and only headings the worklist "needs attention" when the selection IS the stale worklist; otherwise it uses the neutral "selected" heading. Previously okf_audit(stale: false) on a perfectly healthy bundle printed "needs attention (N):" over concepts that were not stale at all -- a factual misstatement the agent would relay verbatim. - Move the freshness token ("stale "/"fresh "/"no-stale-after") into AuditVocabulary.Freshness, and the "--trust"/trust comma-list grammar into AuditVocabulary.TryParseTrustTiers, so neither is spelled out twice between the CLI renderer and the agent tool renderer. The CLI's message on an unknown trust tier is unchanged (still pinned by a CliTests assertion); the tool still returns its own usage message. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01TDM9Aozt9YciNJ3oncrFxG --- src/OKF4net.Agents/OkfBundleTools.cs | 42 +++++++++------ src/OKF4net.Cli/OkfCli.cs | 18 +++---- src/OKF4net/Audit.cs | 50 ++++++++++++++++++ .../OKF4net.Tests/Agents/OkfAuditToolTests.cs | 52 ++++++++++++++++++- tests/OKF4net.Tests/AuditTests.cs | 25 +++++++++ 5 files changed, 158 insertions(+), 29 deletions(-) diff --git a/src/OKF4net.Agents/OkfBundleTools.cs b/src/OKF4net.Agents/OkfBundleTools.cs index db4ac8a..9371fd7 100644 --- a/src/OKF4net.Agents/OkfBundleTools.cs +++ b/src/OKF4net.Agents/OkfBundleTools.cs @@ -465,7 +465,11 @@ public string Search( /// /// Audits the bundle's trust, freshness and lifecycle signals (§5.3–§5.5): /// counts over the whole bundle, then the concepts the filters select, - /// bounded to 20 entries. + /// bounded to 20 entries. The worklist heading tracks : + /// with it true (the default) the selection is the stale worklist, so the + /// heading reads "needs attention"; with it false the selection can include + /// perfectly fresh concepts, so the heading reads the neutral "selected" + /// instead. /// /// Keep only concepts past their stale_after date. /// Comma-separated trust tiers to keep. @@ -481,16 +485,12 @@ public string Audit( HashSet? tiers = null; if (trust is not null) { - tiers = []; - foreach (var entry in trust.Split(',')) + if (!AuditVocabulary.TryParseTrustTiers(trust, out var parsed, out _)) { - if (!AuditVocabulary.TryParseTrustTier(entry.Trim(), out var tier)) - { - return AuditUsageMessage; - } - - tiers.Add(tier); + return AuditUsageMessage; } + + tiers = parsed; } ConceptStatus? parsedStatus = null; @@ -518,7 +518,7 @@ public string Audit( new AuditQuery(stale, tiers, parsedStatus, type), new PinnedClock(Today)); - return RenderAudit(report); + return RenderAudit(report, staleOnly: stale); }); } @@ -1165,7 +1165,17 @@ private static string FormatSearchResults(string query, string? tag, IReadOnlyLi /// 20 findings. Deliberately not shared with the CLI renderer, whose bytes /// are golden-locked and must not move when this string is tuned. /// - private static string RenderAudit(AuditReport report) + /// The audit report to render. + /// + /// Whether the selection IS the stale worklist (the tool's stale + /// parameter). When true, the worklist heading reads needs attention + /// -- otherwise, the selection was narrowed or widened by other filters, so + /// the neutral selected heading is used instead: calling every + /// selected concept a concept that "needs attention" would misstate a + /// selection like stale: false, which can include perfectly fresh + /// concepts. + /// + private static string RenderAudit(AuditReport report, bool staleOnly) { const int MaxResults = 20; var sb = new StringBuilder(); @@ -1190,18 +1200,18 @@ private static string RenderAudit(AuditReport report) sb.Append($"\nstale: {report.StaleCount} of {report.ConceptCount} past stale_after\n"); + var heading = staleOnly ? "needs attention" : "selected"; + if (report.Findings.Count == 0) { - sb.Append("\nneeds attention: none\n"); + sb.Append($"\n{heading}: none\n"); return sb.ToString(); } - sb.Append($"\nneeds attention ({report.Findings.Count}):\n"); + sb.Append($"\n{heading} ({report.Findings.Count}):\n"); foreach (var finding in report.Findings.Take(MaxResults)) { - var freshness = finding.Lifecycle.StaleAfter is { } date - ? (finding.IsStale ? "stale " : "fresh ") + date.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture) - : "no-stale-after"; + var freshness = AuditVocabulary.Freshness(finding.Lifecycle, finding.IsStale); sb.Append(" ") .Append(finding.Id) diff --git a/src/OKF4net.Cli/OkfCli.cs b/src/OKF4net.Cli/OkfCli.cs index 26c0fe1..5f9dbf9 100644 --- a/src/OKF4net.Cli/OkfCli.cs +++ b/src/OKF4net.Cli/OkfCli.cs @@ -395,17 +395,13 @@ private static AuditQuery ParseAuditQuery(string[] args) var trustRaw = FlagValue(args, "--trust"); if (trustRaw is not null) { - tiers = []; - foreach (var entry in trustRaw.Split(',')) + if (!AuditVocabulary.TryParseTrustTiers(trustRaw, out var parsed, out var badEntry)) { - if (!AuditVocabulary.TryParseTrustTier(entry.Trim(), out var tier)) - { - throw new CliOperationException( - $"unknown trust tier \"{entry.Trim()}\"; expected unverified, machine-confirmed or human-reviewed"); - } - - tiers.Add(tier); + throw new CliOperationException( + $"unknown trust tier \"{badEntry}\"; expected unverified, machine-confirmed or human-reviewed"); } + + tiers = parsed; } ConceptStatus? status = null; @@ -431,9 +427,7 @@ private static AuditQuery ParseAuditQuery(string[] args) /// Renders one concept line: id, freshness, trust tier, status -- two spaces between fields. private static string FormatAuditFinding(AuditFinding finding) { - var freshness = finding.Lifecycle.StaleAfter is { } date - ? (finding.IsStale ? "stale " : "fresh ") + date.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture) - : "no-stale-after"; + var freshness = AuditVocabulary.Freshness(finding.Lifecycle, finding.IsStale); return $"{finding.Id} {freshness} {AuditVocabulary.Name(finding.Trust)} {AuditVocabulary.Name(finding.Lifecycle.Status)}"; } diff --git a/src/OKF4net/Audit.cs b/src/OKF4net/Audit.cs index ba14c7a..87da680 100644 --- a/src/OKF4net/Audit.cs +++ b/src/OKF4net/Audit.cs @@ -1,4 +1,6 @@ // SPDX-License-Identifier: LGPL-3.0-or-later +using System.Globalization; + namespace OKF4net; /// @@ -145,6 +147,54 @@ public static bool TryParseStatus(string text, out ConceptStatus status) default: status = ConceptStatus.Stable; return false; } } + + /// + /// Parses a comma-separated list of trust tier names (e.g. + /// "unverified,human-reviewed"). Each entry is trimmed and parsed + /// with ; a duplicate entry is absorbed + /// silently since is a set. The single grammar + /// shared by the CLI's --trust flag and the okf_audit tool's + /// trust parameter -- callers differ only in what they do on + /// failure (the CLI raises a CliOperationException naming + /// ; the tool returns a usage message). + /// + /// The comma-separated list, as typed by the caller. + /// On success, the parsed tiers; empty on failure. + /// On failure, the first entry (trimmed) that failed to parse; on success. + /// when every entry parsed. + public static bool TryParseTrustTiers(string raw, out HashSet tiers, out string? badEntry) + { + tiers = []; + foreach (var entry in raw.Split(',')) + { + var trimmed = entry.Trim(); + if (!TryParseTrustTier(trimmed, out var tier)) + { + tiers = []; + badEntry = trimmed; + return false; + } + + tiers.Add(tier); + } + + badEntry = null; + return true; + } + + /// + /// The freshness token for one concept line (§5.5): "stale <date>" + /// or "fresh <date>" when stale_after parsed, or + /// "no-stale-after" when it is absent or malformed. The single + /// spelling shared by both renderers -- do not spell these tokens as + /// literals outside this method. + /// + /// The concept's lifecycle fields. + /// Whether the concept is stale as of the report's AsOf date. + public static string Freshness(Lifecycle lifecycle, bool isStale) => + lifecycle.StaleAfter is { } date + ? (isStale ? "stale " : "fresh ") + date.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture) + : "no-stale-after"; } /// diff --git a/tests/OKF4net.Tests/Agents/OkfAuditToolTests.cs b/tests/OKF4net.Tests/Agents/OkfAuditToolTests.cs index 77639cb..ab8a9e7 100644 --- a/tests/OKF4net.Tests/Agents/OkfAuditToolTests.cs +++ b/tests/OKF4net.Tests/Agents/OkfAuditToolTests.cs @@ -116,6 +116,56 @@ public void Audit_with_stale_false_and_no_filter_returns_the_whole_corpus() var text = ToolsOver(tmp, new DateOnly(2026, 8, 21)).Audit(stale: false); - Assert.Contains("needs attention (2):", text); + Assert.Contains("selected (2):", text); + } + + /// + /// Regression guard for the bug fixed here: with stale: false, the + /// selection can include perfectly fresh concepts, so calling every one of + /// them "needs attention" would be a factual misstatement the agent could + /// relay verbatim. + /// + [Fact] + public void Audit_with_stale_false_never_says_needs_attention() + { + using var tmp = new TempDir(); + tmp.Write("a.md", "---\ntype: Metric\n---\n"); + tmp.Write("b.md", "---\ntype: Metric\n---\n"); + + var text = ToolsOver(tmp, new DateOnly(2026, 8, 21)).Audit(stale: false); + + Assert.DoesNotContain("needs attention", text); + } + + /// + /// The stale-only path (the tool's default) is the one place "needs + /// attention" is an accurate label -- every selected concept really is + /// past its stale_after date. + /// + [Fact] + public void Audit_stale_only_path_still_says_needs_attention() + { + using var tmp = new TempDir(); + tmp.Write("a.md", "---\ntype: Metric\nstale_after: 2026-01-01\n---\n"); + + var text = ToolsOver(tmp, new DateOnly(2026, 8, 21)).Audit(stale: true); + + Assert.Contains("needs attention (1):", text); + } + + /// + /// The freshness token for a concept with no stale_after at all + /// must come from , same as the CLI + /// -- this line is the tool-side coverage the shared vocabulary lacked. + /// + [Fact] + public void Audit_finding_line_reports_no_stale_after_when_the_field_is_absent() + { + using var tmp = new TempDir(); + tmp.Write("a.md", "---\ntype: Metric\n---\n"); + + var text = ToolsOver(tmp, new DateOnly(2026, 8, 21)).Audit(stale: false); + + Assert.Contains("a no-stale-after unverified stable", text); } } diff --git a/tests/OKF4net.Tests/AuditTests.cs b/tests/OKF4net.Tests/AuditTests.cs index f4874ee..b7360d8 100644 --- a/tests/OKF4net.Tests/AuditTests.cs +++ b/tests/OKF4net.Tests/AuditTests.cs @@ -202,4 +202,29 @@ public void Vocabulary_names_round_trip() Assert.Equal(ConceptStatus.Deprecated, status); Assert.False(AuditVocabulary.TryParseStatus("retired", out _)); } + + /// + /// The single grammar shared by the CLI's --trust flag and the + /// okf_audit tool's trust parameter: comma-split, trim each + /// entry, absorb duplicates (the result is a set), fail on the first + /// unparseable entry and report it (trimmed) via . + /// + [Fact] + public void TryParseTrustTiers_trims_absorbs_duplicates_and_reports_the_bad_entry() + { + Assert.True(AuditVocabulary.TryParseTrustTiers( + "unverified, unverified,human-reviewed", out var tiers, out var badEntry)); + Assert.Equal( + new HashSet { TrustTier.Unverified, TrustTier.HumanReviewed }, + tiers); + Assert.Null(badEntry); + + Assert.False(AuditVocabulary.TryParseTrustTiers( + "unverified,,human-reviewed", out var afterFailure, out var emptyEntry)); + Assert.Empty(afterFailure); + Assert.Equal("", emptyEntry); + + Assert.False(AuditVocabulary.TryParseTrustTiers("bogus", out _, out var unknownEntry)); + Assert.Equal("bogus", unknownEntry); + } } From 6fa9a89df4e636a57f8f27fb04904b4096afc470 Mon Sep 17 00:00:00 2001 From: Julien CHABLE Date: Sat, 22 Aug 2026 07:16:25 +0200 Subject: [PATCH 15/19] docs(audit): clarify what AuditQuery.IsFiltered actually answers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit IsFiltered's doc read as "did the caller pass a filter flag?", but it is true for the CLI's own report-mode query (new AuditQuery(StaleOnly: true)), which no flag produced. Reword it to say precisely what the property answers -- whether the query constrains the selection -- without touching its behavior; All and IsFiltered both stay as mandated by the design spec (§3.1). Pin both ends with a unit test. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01TDM9Aozt9YciNJ3oncrFxG --- src/OKF4net/Audit.cs | 8 +++++++- tests/OKF4net.Tests/AuditTests.cs | 13 +++++++++++++ 2 files changed, 20 insertions(+), 1 deletion(-) diff --git a/src/OKF4net/Audit.cs b/src/OKF4net/Audit.cs index 87da680..8fb4582 100644 --- a/src/OKF4net/Audit.cs +++ b/src/OKF4net/Audit.cs @@ -27,7 +27,13 @@ public readonly record struct AuditQuery( /// The query that keeps every concept. public static AuditQuery All => default; - /// True as soon as one predicate is set. + /// + /// Whether this query constrains the selection below "every concept" -- + /// true whenever any predicate is non-default. This is not "did the + /// caller type a filter flag": the CLI's report mode builds + /// new AuditQuery(StaleOnly: true) itself, with no flag typed, and + /// this is still for it. + /// public bool IsFiltered => StaleOnly || Trust is not null || Status is not null || Type is not null; } diff --git a/tests/OKF4net.Tests/AuditTests.cs b/tests/OKF4net.Tests/AuditTests.cs index b7360d8..163c302 100644 --- a/tests/OKF4net.Tests/AuditTests.cs +++ b/tests/OKF4net.Tests/AuditTests.cs @@ -227,4 +227,17 @@ public void TryParseTrustTiers_trims_absorbs_duplicates_and_reports_the_bad_entr Assert.False(AuditVocabulary.TryParseTrustTiers("bogus", out _, out var unknownEntry)); Assert.Equal("bogus", unknownEntry); } + + /// + /// answers "does this query constrain + /// the selection?", not "did the caller type a flag?" -- the CLI's report + /// mode builds new AuditQuery(StaleOnly: true) itself, with no flag + /// typed, and must still see here. + /// + [Fact] + public void IsFiltered_reflects_whether_the_query_constrains_the_selection() + { + Assert.False(AuditQuery.All.IsFiltered); + Assert.True(new AuditQuery(StaleOnly: true).IsFiltered); + } } From e31d0d6f59f400ea4e215fae0cea8d8234902d54 Mon Sep 17 00:00:00 2001 From: Julien CHABLE Date: Sat, 22 Aug 2026 07:16:35 +0200 Subject: [PATCH 16/19] test(cli): pin positive selection for audit's --type and --status filters Only --status deprecated (empty result) and a positional-order [Theory] that ignores stdout covered these two flags before this, so transposing Type and Status in ParseAuditQuery would leave the whole suite green. Add assertions on tests/fixtures/okf_v02 (2 concepts, both type: Metric, both resolving to status: stable): --type Metric selects both lines, --type metric (lowercase) selects none -- pinning the documented ordinal, case-sensitive rule at the CLI boundary -- and --status stable selects both lines. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01TDM9Aozt9YciNJ3oncrFxG --- tests/OKF4net.Tests/CliTests.cs | 35 +++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/tests/OKF4net.Tests/CliTests.cs b/tests/OKF4net.Tests/CliTests.cs index cb5382f..be1e29a 100644 --- a/tests/OKF4net.Tests/CliTests.cs +++ b/tests/OKF4net.Tests/CliTests.cs @@ -496,6 +496,41 @@ public void Audit_empty_selection_prints_nothing() Assert.Equal("", r.Out); } + /// + /// Positive-selection coverage for --type: both fixture concepts + /// are type: Metric, so the exact-case value must select both, and + /// the lowercase variant must select none -- pinning the documented + /// ordinal, case-sensitive rule (§ Audit.cs AuditQuery.Type) at the + /// CLI boundary. Without this, transposing Type/Status in + /// ParseAuditQuery would leave the suite green. + /// + [Fact] + public void Audit_type_filter_selects_matching_concepts_case_sensitively() + { + var match = Run("audit", V02BundlePath, "--type", "Metric"); + Assert.Equal(0, match.Code); + Assert.Equal(2, match.Out.Split('\n', StringSplitOptions.RemoveEmptyEntries).Length); + + var noMatch = Run("audit", V02BundlePath, "--type", "metric"); + Assert.Equal(0, noMatch.Code); + Assert.Equal("", noMatch.Out); + } + + /// + /// Positive-selection coverage for --status: both fixture concepts + /// resolve to status: stable (one explicitly, one via + /// Lifecycle.From's fallback for the unknown "retired" value), so + /// this must select both. + /// + [Fact] + public void Audit_status_filter_selects_matching_concepts() + { + var r = Run("audit", V02BundlePath, "--status", "stable"); + + Assert.Equal(0, r.Code); + Assert.Equal(2, r.Out.Split('\n', StringSplitOptions.RemoveEmptyEntries).Length); + } + [Fact] public void Audit_three_tier_idiom_returns_every_concept() { From b6e874ea4b23c931eb48da6eab31424baf21878a Mon Sep 17 00:00:00 2001 From: Julien CHABLE Date: Sat, 22 Aug 2026 07:16:53 +0200 Subject: [PATCH 17/19] docs(audit): fix tool-count/wording drift and record audit shipping - README.md: GetTools() returns eleven tools unconditionally (was "ten"), plus a twelfth -- okf_run_computation -- conditional on an attestation orchestrator, matching the table 18 lines below which already said eleven/twelfth. Also quote the CLI's own --help wording for the audit verb line ("Report trust, freshness and lifecycle across the bundle") instead of paraphrasing it, matching every other verb line in that block. - src/OKF4net.Agents/README.md: "The nine tools" was stale even before this branch and is now three tools behind (okf_audit, okf_get_computation, and the conditional okf_run_computation). Correct the count and list to match GetTools(). - CLAUDE.md: add audit to the CLI verb list, and add one line under src/OKF4net/ pointing at ConceptAudit as the single shared corpus-level query behind okf audit and okf_audit -- the same "do not fork this" guidance already given for ConceptSearch. - ROADMAP.md: record that okf audit shipped, in the same style as the "static render shipped" bundle-viewer entry. - CHANGELOG.md: bold the okf audit entry to match its feature-scale neighbors (**okf render ...**, **okf validate/okf info gain a --json flag**), and remove the blank line that had made the Added list loose around it. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01TDM9Aozt9YciNJ3oncrFxG --- CHANGELOG.md | 3 +-- CLAUDE.md | 3 ++- README.md | 6 +++--- ROADMAP.md | 6 ++++++ src/OKF4net.Agents/README.md | 8 +++++--- 5 files changed, 17 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2e4db9e..0bf11da 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,12 +10,11 @@ and this project adheres to ### Added -- `okf audit` — a corpus-level query over a bundle's trust (§5.3), lifecycle +- **`okf audit`** — a corpus-level query over a bundle's trust (§5.3), lifecycle (§5.4) and staleness (§5.5) signals: counts plus a filterable worklist, with `--stale`, `--trust`, `--status`, `--type`, `--as-of` and `--json`. Backed by the new `ConceptAudit` in the core library and exposed to agents as the read-only `okf_audit` tool. - - **`okf render --out `** generates a self-contained, browsable HTML site from a bundle: one page per concept (frontmatter table + rendered body), a generated index, navigable cross-links with broken links flagged, diff --git a/CLAUDE.md b/CLAUDE.md index cbb3bc9..4225381 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -33,10 +33,11 @@ Requires .NET SDK 10.0+. CI (ci.yml) runs build+test on Linux/Windows/macOS, `do - **`src/OKF4net/`** — the library. One file per spec concern, following the OKF reference implementation's structure: `ConceptId` (§2), `Bundle` (§3, permissive loading — parse failures go into `Bundle.ParseErrors`, never abort), `OkfDocument`/`Frontmatter` (§4), `Links.cs`/`LinkScanner` (§6, legacy citations §13.1), `IndexGenerator` (§8), `ChangeLog` (§9), `Validate.cs`/`BundleValidator` (§11). The README has the full spec-section → type mapping table. - `ConceptSearch` — the single shared full-text scorer (title x3, tags/description x2, body x1) used by both `OKF4net.Agents` (`okf_search`/context provider) and `OKF4net.Catalog` (`OkfBundleKnowledgeSource`, `FileMemoryStore`); do not fork a second scorer in either consumer. + - `Audit.cs` — `ConceptAudit`, the single shared corpus-level query behind both `okf audit` and the `okf_audit` tool; the two renderers are deliberately separate (the CLI's bytes are golden-locked), but the computation and the `AuditVocabulary` labels must not be forked. - `Yaml/` — the documented YAML *subset* (scalars, lists, shallow maps, block/flow, `|`/`>`); it deliberately rejects anchors/tags/multi-docs with clear errors. `Frontmatter` wraps an order-preserving `YamlMapping` with typed getters rather than a fixed DTO, so unknown producer keys survive round-trips. - `Internal/LfLines.cs` — the single shared line splitter (splits on `\n` only, stripping a preceding `\r`). Use it anywhere `\n`-based line splitting matters; do not reintroduce private copies. - `Internal/ReparsePoints.cs` — internal symlink/junction detection; `OKF4net.Catalog` is granted `InternalsVisibleTo` so it can reuse this seam rather than duplicating a platform-specific implementation. -- **`src/OKF4net.Cli/`** — the `okf` binary (`validate`/`info`/`index`/`graph`/`parse`/`fmt`/`render`), published Native AOT (`PublishAot`, `InvariantGlobalization`). All logic lives in `OkfCli.Run(args, out, err)` so tests invoke it in-process without spawning a process. +- **`src/OKF4net.Cli/`** — the `okf` binary (`validate`/`audit`/`info`/`index`/`graph`/`parse`/`fmt`/`render`), published Native AOT (`PublishAot`, `InvariantGlobalization`). All logic lives in `OkfCli.Run(args, out, err)` so tests invoke it in-process without spawning a process. - **`src/OKF4net.Attestation/`** — zero-dep §10 attested-computation orchestration, referencing only `OKF4net`. Defines the host-plugged contracts (`IParameterBinder`, `IComputationExecutor`, `IAttester`, resolved per concept's `runtime` field through `IAttestationRuntimeRegistry`) and the value types that flow between them (`BoundComputation`, `Receipt`, `AttestationVerdict`, `AttestationContext`, `AttestationOutcome`); `AttestationOrchestrator.RunAsync` drives one run end to end (resolve → bind → execute → receipt-shape check → attest → gate on verdict + `stale_after`), errors-as-data, never writing a verdict back to the bundle (§10.6). Referenced by `OKF4net.Agents` to back `okf_run_computation`. - **`src/OKF4net.Agents/`** — Microsoft Agent Framework layer exposing OKF bundle operations as function tools (e.g. `OkfBundleTools`) plus `OkfContextProvider`, an `AIContextProvider` that auto-injects budget-bounded bundle context and captures deterministic per-day memory concepts; the only project depending on `Microsoft.Agents.AI`. - **`src/OKF4net.Catalog/`** — knowledge-catalog model and logic, referencing only `OKF4net` (BCL otherwise; zero `PackageReference`). Depended on by `OKF4net.Catalog.Hosting`. Each manifest source carries a `role` (`SourceRole`): `Knowledge` (read-only, searched by `IKnowledgeResolver`) or `Memory` (writable, scoped by a required `tier` — `session`/`user`/`tenant`, all three backed by `FileMemoryStore`, fed by `IMemoryStore`, never searched by the resolver); any other `role` string in `catalog.json` is rejected (`CatalogDiagnosticCode.IllegalRole`). diff --git a/README.md b/README.md index 9e72896..8e5fda2 100644 --- a/README.md +++ b/README.md @@ -176,7 +176,7 @@ On any OS, build from source — see [Building & testing](#building--testing). ``` okf validate Check a bundle against OKF v0.2 conformance (§11) -okf audit Query trust, freshness and lifecycle signals (§5.3–§5.5) +okf audit Report trust, freshness and lifecycle across the bundle okf info Summarize a bundle (concepts, types, links, version) okf index (Re)generate every index.md in the bundle okf graph Print the cross-link graph (--dot for Graphviz DOT) @@ -229,8 +229,8 @@ machine. Full command reference with real output samples: `src/OKF4net.Agents/` exposes bundle operations as function tools for the [Microsoft Agent Framework](https://github.com/microsoft/agent-framework): `OkfBundleTools` wraps one bundle root and its `GetTools()` method returns -ten ready-to-use `AITool`s unconditionally, which `AsAIAgent` turns into an -agent's tool list, plus an eleventh — `okf_run_computation` — only when the +eleven ready-to-use `AITool`s unconditionally, which `AsAIAgent` turns into an +agent's tool list, plus a twelfth — `okf_run_computation` — only when the tool set is constructed with an `OKF4net.Attestation` orchestrator wired in (see [Attested computation](#attested-computation-okf4netattestation)). diff --git a/ROADMAP.md b/ROADMAP.md index cd0cba3..fabf3a0 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -14,6 +14,12 @@ are the concrete entry points. ## Next +- **`okf audit` shipped** — a corpus-level query over a bundle's trust (§5.3), + lifecycle (§5.4) and staleness (§5.5) signals: counts plus a filterable + worklist, across the CLI verb and the read-only `okf_audit` agent tool, + backed by the shared `ConceptAudit`/`AuditVocabulary` model in `OKF4net`. + Motivated by ["OKF v0.2 Quietly Admits the Folder Has a Ceiling"](https://medium.com/@davidroliver/okf-v0-2-quietly-admits-the-folder-has-a-ceiling-the-way-up-is-a-library-25fa54e872f9) + — see [its design spec](docs/superpowers/specs/2026-08-21-okf-audit-design.md). - More `OKF4net.Agents` samples with Microsoft Agent Framework — the first, `samples/acme-retail-agent`, shipped in 0.4.0; more welcome. - `OKF4net.Catalog` samples: `samples/catalog-explorer` (multi-source diff --git a/src/OKF4net.Agents/README.md b/src/OKF4net.Agents/README.md index 70fe308..f65e2c3 100644 --- a/src/OKF4net.Agents/README.md +++ b/src/OKF4net.Agents/README.md @@ -26,11 +26,13 @@ AIAgent agent = chatClient.AsAIAgent( var response = await agent.RunAsync("Summarize the concepts in this bundle."); ``` -## The nine tools +## The eleven tools -`okf_read_concept`, `okf_browse`, `okf_graph`, `okf_search`, +`okf_read_concept`, `okf_browse`, `okf_graph`, `okf_search`, `okf_audit`, `okf_write_concept`, `okf_append_log`, `okf_regenerate_indexes`, -`okf_validate_bundle`, `okf_changes_since`. +`okf_validate_bundle`, `okf_changes_since`, `okf_get_computation` — plus a +twelfth, `okf_run_computation`, only when the tool set is constructed with an +`OKF4net.Attestation` orchestrator wired in. All tools return agent-friendly markdown/plain text and never throw for expected errors (unknown ids, invalid paths, malformed input) — the agent From e0f50c3bf0e0ccf38a9f3b5d90b55afa9d3cf152 Mon Sep 17 00:00:00 2001 From: Julien CHABLE Date: Sat, 22 Aug 2026 08:56:26 +0200 Subject: [PATCH 18/19] test(agents): drive okf_audit through the real agent loop MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every other okf_audit test calls the C# method directly, so nothing covered the framework's JSON argument binding: a model passes `stale` as a boolean and `trust` as a comma-separated string, and the binding has to reach the method's parameters for the filters to take effect. The bundle is built inline with the UtcNow seam pinned, so one concept is stale AND unverified while another is equally stale but human-reviewed. A binding failure that dropped the trust argument would list both — verified by mutating the script's trust argument to null and watching the test fail, then restoring it. Co-Authored-By: Claude Opus 5 (1M context) --- .../Agents/AgentIntegrationTests.cs | 72 +++++++++++++++++++ 1 file changed, 72 insertions(+) diff --git a/tests/OKF4net.Tests/Agents/AgentIntegrationTests.cs b/tests/OKF4net.Tests/Agents/AgentIntegrationTests.cs index 02fde3e..ea7a613 100644 --- a/tests/OKF4net.Tests/Agents/AgentIntegrationTests.cs +++ b/tests/OKF4net.Tests/Agents/AgentIntegrationTests.cs @@ -126,4 +126,76 @@ public async Task Agent_writes_concept_regenerates_indexes_and_validates_end_to_ Assert.Contains("✓ conformant", scriptedClient.ObservedFunctionResults[2]); Assert.DoesNotContain("✗", scriptedClient.ObservedFunctionResults[2]); } + + /// + /// The read-only counterpart of the write scenario, and the question the + /// audit surface exists for: "which concepts are past their stale_after + /// date and were never verified by a human?". It is the only test that + /// drives okf_audit through the framework's real function-invoking + /// pipeline rather than calling the C# method directly, so it is what + /// proves the JSON argument binding works: the scripted "model" passes a + /// boolean and a comma-separated tier list as JSON, and the framework + /// must bind them to the method's bool and string? + /// parameters for the filter to take effect. + /// + /// The bundle is built inline so the observation date can be pinned via + /// the UtcNow seam: metrics/orphan is stale AND unverified + /// (the answer), while metrics/dau is equally stale but + /// human-reviewed (the concept the trust filter must exclude). A binding + /// failure that silently dropped the trust argument would return + /// both and fail here. + /// + [Fact] + public async Task Agent_audits_the_bundle_for_stale_never_human_verified_concepts() + { + using var tmp = new TempDir(); + tmp.Write( + "metrics/dau.md", + "---\ntype: Metric\ntitle: Daily Active Users\nstale_after: 2026-01-01\n" + + "verified:\n - { by: human:ada, at: 2026-01-01T00:00:00Z }\n---\n"); + tmp.Write( + "metrics/orphan.md", + "---\ntype: Metric\ntitle: Orphaned Metric\nstale_after: 2026-01-01\n---\n"); + + var tools = new OkfBundleTools(tmp.Path) + { + UtcNow = () => new DateTime(2026, 8, 21, 0, 0, 0, DateTimeKind.Utc), + }; + + const string auditAnswer = "One concept is stale and was never verified by a human: metrics/orphan."; + + var scriptedClient = new ScriptedChatClient( + [ + ScriptStep.Call("okf_audit", new Dictionary + { + ["stale"] = true, + ["trust"] = "unverified,machine-confirmed", + }), + ScriptStep.Answer(auditAnswer), + ]); + + AIAgent agent = scriptedClient.AsAIAgent(tools: tools.GetTools()); + + var response = await agent.RunAsync( + "Quels concepts ont dépassé leur stale_after sans avoir jamais été vérifiés par un humain ?"); + + Assert.Equal(auditAnswer, response.Text); + + // Two round-trips: the tool-call turn and the final-answer turn. + Assert.Equal(2, scriptedClient.TurnsTaken); + + var auditResult = Assert.Single(scriptedClient.ObservedFunctionResults); + + // The filters bound and took effect: the stale, unverified concept is + // listed under the worklist heading; the stale but human-reviewed one + // is not. + Assert.Contains("needs attention (1):", auditResult); + Assert.Contains("metrics/orphan stale 2026-01-01 unverified", auditResult); + Assert.DoesNotContain("metrics/dau", auditResult); + + // The counters still describe the whole bundle, not the selection -- + // the distinction the audit surface is built on. + Assert.Contains("concepts: 2", auditResult); + Assert.Contains("stale: 2 of 2 past stale_after", auditResult); + } } From 72fefe6b9e05e274226afbb9886aad37269c4ba7 Mon Sep 17 00:00:00 2001 From: Julien CHABLE Date: Sat, 22 Aug 2026 14:14:19 +0200 Subject: [PATCH 19/19] docs(sample): teach the acme-retail agent to ask corpus-level questions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit okf_audit was already reaching the sample's agent — Program.cs filters GetTools() by WriteToolNames, so a new read-only tool is exposed with no code change. What was missing is that nothing told the agent, or the reader, that a corpus-level question is now answerable in one call: a model asked "what is stale?" would browse concept by concept, which is the very behaviour the audit surface exists to replace. Adds one sentence to the system instructions, a "Questions worth asking" section splitting retrieval questions from corpus-level ones, and a note in "What it does" on how the two use the tools differently. The section is honest about the demo: nothing in this bundle is stale yet, so the trust question is the one that bites today, and the bundle starts reporting seven stale concepts on 2027-01-01. Also realigns the sample's Microsoft.Agents.AI pin from 1.15.0 to 1.17.0. That break predates this branch: a dependabot bump moved OKF4net.Agents to 1.17.0 and the sample's own pin stayed put, which NU1605 rejects as a downgrade. Nothing caught it because samples/ is outside OKF4net.sln and outside CI — the sample could not be built at all before this commit. Co-Authored-By: Claude Opus 5 (1M context) --- samples/acme-retail-agent/README.md | 33 +++++++++++++++++++ .../AcmeRetailAgent/AcmeRetailAgent.csproj | 2 +- .../src/AcmeRetailAgent/Program.cs | 5 ++- 3 files changed, 38 insertions(+), 2 deletions(-) diff --git a/samples/acme-retail-agent/README.md b/samples/acme-retail-agent/README.md index ec9202e..6fbaa55 100644 --- a/samples/acme-retail-agent/README.md +++ b/samples/acme-retail-agent/README.md @@ -51,6 +51,32 @@ OKF_CHAT_BASE_URL=http://localhost:11434/v1 OKF_CHAT_MODEL=llama3 \ (or pipe a prompt via stdin instead of `--prompt`). Type `exit` or `quit` to leave the interactive REPL. +## Questions worth asking + +Grounded answers about one concept — these go through +`okf_search`/`okf_read_concept`: + +- *What is Acme's FY2026 revenue recognition policy?* +- *How is gross margin defined, and what does it depend on?* + +Questions about the bundle **as a whole** — these go through `okf_audit`, in +one call rather than by opening concepts one at a time: + +- *Which concepts have never been verified by a human?* — the interesting + one today: eight of the nine concepts carry a `human:` verifier, so this + isolates `skills/run-on-bq`. +- *How healthy is this knowledge base — how much of it is human-reviewed, + and how much is stale?* +- *Is anything deprecated?* + +Note on the freshness question specifically: no concept in this bundle is +past its `stale_after` date *yet*, so "what is stale?" correctly answers +"nothing" today. Most concepts carry `stale_after: 2026-12-31`, so this +sample starts reporting seven stale concepts on 2027-01-01 — which is the +point the bundle is making, not a bug in it. To see the stale path before +then, the CLI can pin the date: `okf audit bundles/acme_retail --as-of +2027-06-01`. + ## What it does Wires `OkfBundleTools` (constructed without an `AttestationOrchestrator`, so @@ -62,6 +88,13 @@ interactive mode keeps one `AgentSession` across turns; one-shot mode runs a single turn and exits. Each response prints a `[tools: ...]` line naming any `okf_*` tools the agent called, for visibility into what it did. +Two kinds of question are demonstrated, and they use the tools differently. +Retrieval questions (`okf_search`, `okf_read_concept`, `okf_graph`) pull one +concept, or a few. Corpus-level questions about trust, freshness and +lifecycle go to `okf_audit`, which answers them in one call by reading every +concept's §5.3–§5.5 frontmatter — the `[tools: ...]` line is what shows you +which path the model actually took. + "Read-only" is enforced by construction, not just by convention: `Program.cs` filters `okf_write_concept`, `okf_append_log`, and `okf_regenerate_indexes` out of the tool list before it ever reaches the diff --git a/samples/acme-retail-agent/src/AcmeRetailAgent/AcmeRetailAgent.csproj b/samples/acme-retail-agent/src/AcmeRetailAgent/AcmeRetailAgent.csproj index 00d08b4..6dcd460 100644 --- a/samples/acme-retail-agent/src/AcmeRetailAgent/AcmeRetailAgent.csproj +++ b/samples/acme-retail-agent/src/AcmeRetailAgent/AcmeRetailAgent.csproj @@ -8,7 +8,7 @@ - + diff --git a/samples/acme-retail-agent/src/AcmeRetailAgent/Program.cs b/samples/acme-retail-agent/src/AcmeRetailAgent/Program.cs index 4fb2949..027f232 100644 --- a/samples/acme-retail-agent/src/AcmeRetailAgent/Program.cs +++ b/samples/acme-retail-agent/src/AcmeRetailAgent/Program.cs @@ -32,7 +32,10 @@ + "the okf_* tools to answer questions -- do not guess at bundle " + "content. Attested Computations can be inspected with " + "okf_get_computation (their contract and sanctioned SQL) but this " - + "sample cannot run them."; + + "sample cannot run them. Questions about the bundle as a whole -- " + + "which concepts are stale, unverified, deprecated, or of a given type -- " + + "are answered by a single okf_audit call, never by reading concepts " + + "one at a time."; var tools = new OkfBundleTools(bundleRoot); var contextProvider = new OkfContextProvider(tools);