diff --git a/CLAUDE.md b/CLAUDE.md index eebe3d5..4b36b8f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -142,7 +142,7 @@ deliberate commit, not a build-time artifact; the matrix is citable from outside ## Scorecard JSON fields -`src/scorecard/mod.rs` emits `schema_version: "0.5"`. The schema evolves additively during the `0.x` pre-launch window; +`src/scorecard/mod.rs` emits `schema_version: "0.8"`. The schema evolves additively during the `0.x` pre-launch window; consumers feature-detect each addition rather than pinning exact shape. Cumulative history: - `0.2`: `coverage_summary` (three-way `{must, should, may} × {total, verified}` counts), `audience`, `audit_profile`. @@ -150,6 +150,12 @@ consumers feature-detect each addition rather than pinning exact shape. Cumulati - `0.4`: four top-level objects making the scorecard self-describing: `tool`, `anc`, `run`, `target`. - `0.5`: `badge` block surfacing agent-native badge eligibility, embed snippet, and badge/scorecard URLs derived from the live run. +- `0.6`: 7-status taxonomy (`opt_out` and `n_a` added to `status`), matching counters in `summary`, `tier` on each + result, one result per requirement row instead of per-`audit_id`, antecedent propagation for conditional rows. +- `0.7`: reserved bump for the role-based JSON-error-envelope validator reframe (PR #79). Shape unchanged from `0.6`. +- `0.8`: optional `using_domain_verbs: bool` and `domain_match_count: usize` on each result row, populated when + `p6-standard-names` Passes via per-CLI `.anc.toml [p6] domain_verbs` recognition; Pass evidence string is populated + with the built-in vs domain ratio. Fields are absent (not `null`) from rows that did not consult `domain_verbs`. Existing field semantics: @@ -202,10 +208,29 @@ Existing field semantics: appends a post-summary hint via `BadgeInfo::text_hint()` when `eligible`; the same `tool.name` is used for the slug so the JSON `embed_markdown` and the printed hint can never disagree. +`0.8` addition (`MitigationInfo` carrier on `AuditResult`): + +- `MitigationInfo { using_domain_verbs, domain_match_count, domain_match_examples, builtin_match_count, subcommand_total + }` is attached to an `AuditResult` when the audit's verdict was assisted by a documented per-CLI opt-in. Today's only + producer is `src/audits/behavioral/standard_names.rs`: when `p6-standard-names` Passes because one or more subcommands + were recognized via `.anc.toml [p6] domain_verbs` (rather than the built-in `STANDARD_VERBS` list), the audit fills + `MitigationInfo` with the bifurcated match counts and the first `DOMAIN_MATCH_EXAMPLES_LIMIT` (5) matched domain-verb + names in encounter order. +- `AuditResultView` surfaces two top-level fields derived from the carrier: `using_domain_verbs: Option` and + `domain_match_count: Option`. Both use `skip_serializing_if = "Option::is_none"` so they are absent from rows + that did not consult `domain_verbs`. The Pass row's `evidence` field is populated (rather than `null`) via + `format_pass_evidence(&mitigation)`; rows without mitigation keep the historical `evidence: null` on Pass. +- The carrier shape is deliberately not audit-specific. Future audits that admit per-CLI mitigation (suppression profile + assistance, conditional-applicability config) can populate `MitigationInfo` with the same fields rather than growing + parallel typed carriers. The semantic contract is "this verdict depended on a self-declared opt-in; here is what + assisted." + Always-present null contract: `tool.version`, `tool.binary`, `target.path`, `target.command` serialize as JSON `null` when not applicable, never as missing keys. Consumers can access these paths unconditionally. The exception is `audience_reason`, which uses `skip_serializing_if = "Option::is_none"`; its absence carries information (audience has a -label). +label). The `0.8` `using_domain_verbs` / `domain_match_count` fields follow the `audience_reason` pattern (absent when +not applicable) — *not* the `tool.version` always-present-null pattern — because their absence is itself the signal that +no mitigation was needed. Consumers (notably the site's `/score/` page) must feature-detect the new fields, since pre-`0.4` scorecards lack the four metadata blocks; pre-`0.5` scorecards lack `badge`. The site's `agentnative-site/registry.yaml` will eventually diff --git a/schema/scorecard.schema.json b/schema/scorecard.schema.json index 4983f8b..6b39c05 100644 --- a/schema/scorecard.schema.json +++ b/schema/scorecard.schema.json @@ -1,6 +1,6 @@ { "$schema": "https://json-schema.org/draft/2020-12/schema", - "$id": "https://anc.dev/scorecard-v0.7.schema.json", + "$id": "https://anc.dev/scorecard-v0.8.schema.json", "title": "agentnative scorecard", "description": "JSON Schema for `anc audit --output json` scorecards (schema version 0.7). Schema 0.7 introduces the 7-status taxonomy (`opt_out` and `n_a` added to `status`, with matching counters in `summary`), per-row emission (one result per requirement-row instead of per audit_id; `tier` and `audit_id` fields added to each result), and antecedent propagation for conditional requirements. See docs/plans/2026-05-21-001-feat-scorecard-fairness-taxonomy-plan.md in the agentnative-site repo for the full taxonomy rationale.", "type": "object", @@ -23,7 +23,7 @@ "schema_version": { "type": "string", "description": "Scorecard schema version. Pre-launch additive — consumers feature-detect new fields rather than pin to an exact value.", - "examples": ["0.7"] + "examples": ["0.8"] }, "results": { "type": "array", @@ -214,7 +214,7 @@ }, "examples": [ { - "schema_version": "0.7", + "schema_version": "0.8", "results": [ { "id": "p1-must-no-interactive", diff --git a/src/audits/behavioral/about_long_about.rs b/src/audits/behavioral/about_long_about.rs index a7f813d..9021a0f 100644 --- a/src/audits/behavioral/about_long_about.rs +++ b/src/audits/behavioral/about_long_about.rs @@ -66,6 +66,7 @@ impl Audit for AboutLongAboutAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/behavioral/actionable_errors.rs b/src/audits/behavioral/actionable_errors.rs index 33b227a..dc0482d 100644 --- a/src/audits/behavioral/actionable_errors.rs +++ b/src/audits/behavioral/actionable_errors.rs @@ -69,6 +69,7 @@ impl Audit for ActionableErrorsAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/behavioral/auto_verbosity.rs b/src/audits/behavioral/auto_verbosity.rs index 2551d48..36a30cc 100644 --- a/src/audits/behavioral/auto_verbosity.rs +++ b/src/audits/behavioral/auto_verbosity.rs @@ -79,6 +79,7 @@ impl Audit for AutoVerbosityAudit { layer: self.layer(), status, confidence: Confidence::Low, + mitigation: None, }) } } diff --git a/src/audits/behavioral/bad_args.rs b/src/audits/behavioral/bad_args.rs index dc3d300..f3372fe 100644 --- a/src/audits/behavioral/bad_args.rs +++ b/src/audits/behavioral/bad_args.rs @@ -55,6 +55,7 @@ impl Audit for BadArgsAudit { layer: AuditLayer::Behavioral, status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/behavioral/bundle_install.rs b/src/audits/behavioral/bundle_install.rs index beffb0b..bc0161f 100644 --- a/src/audits/behavioral/bundle_install.rs +++ b/src/audits/behavioral/bundle_install.rs @@ -54,6 +54,7 @@ impl Audit for BundleInstallAudit { layer: self.layer(), status: AuditStatus::Pass, confidence: Confidence::High, + mitigation: None, }); } @@ -69,6 +70,7 @@ impl Audit for BundleInstallAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/behavioral/bundle_update.rs b/src/audits/behavioral/bundle_update.rs index 7a8597e..7538a6f 100644 --- a/src/audits/behavioral/bundle_update.rs +++ b/src/audits/behavioral/bundle_update.rs @@ -49,6 +49,7 @@ impl Audit for BundleUpdateAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/behavioral/color_flag.rs b/src/audits/behavioral/color_flag.rs index 0a2a5d7..310a36d 100644 --- a/src/audits/behavioral/color_flag.rs +++ b/src/audits/behavioral/color_flag.rs @@ -48,6 +48,7 @@ impl Audit for ColorFlagAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/behavioral/consistent_envelope.rs b/src/audits/behavioral/consistent_envelope.rs index f63982d..aa13f3a 100644 --- a/src/audits/behavioral/consistent_envelope.rs +++ b/src/audits/behavioral/consistent_envelope.rs @@ -82,6 +82,7 @@ impl Audit for ConsistentEnvelopeAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/behavioral/consistent_naming.rs b/src/audits/behavioral/consistent_naming.rs index c4fcbf6..f923936 100644 --- a/src/audits/behavioral/consistent_naming.rs +++ b/src/audits/behavioral/consistent_naming.rs @@ -132,6 +132,7 @@ impl Audit for ConsistentNamingAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/behavioral/cursor_pagination.rs b/src/audits/behavioral/cursor_pagination.rs index 5bcd01f..10e468d 100644 --- a/src/audits/behavioral/cursor_pagination.rs +++ b/src/audits/behavioral/cursor_pagination.rs @@ -52,6 +52,7 @@ impl Audit for CursorPaginationAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/behavioral/defaults_in_help.rs b/src/audits/behavioral/defaults_in_help.rs index 67cd34f..c26b860 100644 --- a/src/audits/behavioral/defaults_in_help.rs +++ b/src/audits/behavioral/defaults_in_help.rs @@ -50,6 +50,7 @@ impl Audit for DefaultsInHelpAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/behavioral/env_hints.rs b/src/audits/behavioral/env_hints.rs index fe90f6e..95599ee 100644 --- a/src/audits/behavioral/env_hints.rs +++ b/src/audits/behavioral/env_hints.rs @@ -53,6 +53,7 @@ impl Audit for EnvHintsAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/behavioral/examples_subcommand.rs b/src/audits/behavioral/examples_subcommand.rs index ba508e7..ef68840 100644 --- a/src/audits/behavioral/examples_subcommand.rs +++ b/src/audits/behavioral/examples_subcommand.rs @@ -48,6 +48,7 @@ impl Audit for ExamplesSubcommandAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/behavioral/flag_existence.rs b/src/audits/behavioral/flag_existence.rs index bc0e6a0..af21e32 100644 --- a/src/audits/behavioral/flag_existence.rs +++ b/src/audits/behavioral/flag_existence.rs @@ -84,6 +84,7 @@ impl Audit for FlagExistenceAudit { .into(), ), confidence: Confidence::High, + mitigation: None, }); } @@ -113,6 +114,7 @@ impl Audit for FlagExistenceAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/behavioral/force_yes.rs b/src/audits/behavioral/force_yes.rs index 85c860f..767ccd2 100644 --- a/src/audits/behavioral/force_yes.rs +++ b/src/audits/behavioral/force_yes.rs @@ -74,6 +74,7 @@ impl Audit for ForceYesAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/behavioral/help.rs b/src/audits/behavioral/help.rs index cdc171b..fba1b0a 100644 --- a/src/audits/behavioral/help.rs +++ b/src/audits/behavioral/help.rs @@ -64,6 +64,7 @@ impl Audit for HelpAudit { layer: AuditLayer::Behavioral, status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/behavioral/install_all.rs b/src/audits/behavioral/install_all.rs index 2196eb9..1fbe8ee 100644 --- a/src/audits/behavioral/install_all.rs +++ b/src/audits/behavioral/install_all.rs @@ -51,6 +51,7 @@ impl Audit for InstallAllAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/behavioral/json_aliases.rs b/src/audits/behavioral/json_aliases.rs index 50d3adf..1be936e 100644 --- a/src/audits/behavioral/json_aliases.rs +++ b/src/audits/behavioral/json_aliases.rs @@ -53,6 +53,7 @@ impl Audit for JsonAliasesAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/behavioral/json_error_output.rs b/src/audits/behavioral/json_error_output.rs index 1734103..15a2d3f 100644 --- a/src/audits/behavioral/json_error_output.rs +++ b/src/audits/behavioral/json_error_output.rs @@ -66,6 +66,7 @@ impl Audit for JsonErrorOutputAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/behavioral/json_errors.rs b/src/audits/behavioral/json_errors.rs index 66a517a..eb008b8 100644 --- a/src/audits/behavioral/json_errors.rs +++ b/src/audits/behavioral/json_errors.rs @@ -261,6 +261,7 @@ impl Audit for JsonErrorsAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/behavioral/json_output.rs b/src/audits/behavioral/json_output.rs index 7c8e68b..ce0dc14 100644 --- a/src/audits/behavioral/json_output.rs +++ b/src/audits/behavioral/json_output.rs @@ -60,6 +60,7 @@ impl Audit for JsonOutputAudit { layer: AuditLayer::Behavioral, status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/behavioral/limit_flag.rs b/src/audits/behavioral/limit_flag.rs index 10a8ab0..e7c2ed7 100644 --- a/src/audits/behavioral/limit_flag.rs +++ b/src/audits/behavioral/limit_flag.rs @@ -53,6 +53,7 @@ impl Audit for LimitFlagAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/behavioral/mod.rs b/src/audits/behavioral/mod.rs index 0ffd4c2..504ea84 100644 --- a/src/audits/behavioral/mod.rs +++ b/src/audits/behavioral/mod.rs @@ -35,7 +35,7 @@ mod rich_tui; mod schema_print; mod secret_non_leaky_path; mod sigpipe; -mod standard_names; +pub(crate) mod standard_names; mod stdin_input; mod structured_exit_codes; mod subcommand_examples; diff --git a/src/audits/behavioral/more_formats.rs b/src/audits/behavioral/more_formats.rs index db88fba..880ed89 100644 --- a/src/audits/behavioral/more_formats.rs +++ b/src/audits/behavioral/more_formats.rs @@ -54,6 +54,7 @@ impl Audit for MoreFormatsAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/behavioral/no_color.rs b/src/audits/behavioral/no_color.rs index f4b13a2..e63a679 100644 --- a/src/audits/behavioral/no_color.rs +++ b/src/audits/behavioral/no_color.rs @@ -56,6 +56,7 @@ impl Audit for NoColorBehavioralAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/behavioral/no_pager_behavioral.rs b/src/audits/behavioral/no_pager_behavioral.rs index a01e5f3..e48a82a 100644 --- a/src/audits/behavioral/no_pager_behavioral.rs +++ b/src/audits/behavioral/no_pager_behavioral.rs @@ -57,6 +57,7 @@ impl Audit for NoPagerBehavioralAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/behavioral/non_interactive.rs b/src/audits/behavioral/non_interactive.rs index 275eb26..5a246ef 100644 --- a/src/audits/behavioral/non_interactive.rs +++ b/src/audits/behavioral/non_interactive.rs @@ -107,6 +107,7 @@ impl Audit for NonInteractiveAudit { layer: AuditLayer::Behavioral, status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/behavioral/paired_examples.rs b/src/audits/behavioral/paired_examples.rs index 061d247..e634373 100644 --- a/src/audits/behavioral/paired_examples.rs +++ b/src/audits/behavioral/paired_examples.rs @@ -69,6 +69,7 @@ impl Audit for PairedExamplesAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/behavioral/quiet.rs b/src/audits/behavioral/quiet.rs index 87d5135..21710bf 100644 --- a/src/audits/behavioral/quiet.rs +++ b/src/audits/behavioral/quiet.rs @@ -53,6 +53,7 @@ impl Audit for QuietAudit { layer: AuditLayer::Behavioral, status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/behavioral/raw_flag.rs b/src/audits/behavioral/raw_flag.rs index 168b835..5ae3ee2 100644 --- a/src/audits/behavioral/raw_flag.rs +++ b/src/audits/behavioral/raw_flag.rs @@ -48,6 +48,7 @@ impl Audit for RawFlagAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/behavioral/read_write_distinction.rs b/src/audits/behavioral/read_write_distinction.rs index 3459f98..34ef0b2 100644 --- a/src/audits/behavioral/read_write_distinction.rs +++ b/src/audits/behavioral/read_write_distinction.rs @@ -61,6 +61,7 @@ impl Audit for ReadWriteDistinctionAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/behavioral/rich_tui.rs b/src/audits/behavioral/rich_tui.rs index 8105ca5..f312b02 100644 --- a/src/audits/behavioral/rich_tui.rs +++ b/src/audits/behavioral/rich_tui.rs @@ -65,6 +65,7 @@ impl Audit for RichTuiAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/behavioral/schema_print.rs b/src/audits/behavioral/schema_print.rs index d3c656f..3b752fd 100644 --- a/src/audits/behavioral/schema_print.rs +++ b/src/audits/behavioral/schema_print.rs @@ -76,6 +76,7 @@ impl Audit for SchemaPrintAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/behavioral/secret_non_leaky_path.rs b/src/audits/behavioral/secret_non_leaky_path.rs index 436d340..79016bc 100644 --- a/src/audits/behavioral/secret_non_leaky_path.rs +++ b/src/audits/behavioral/secret_non_leaky_path.rs @@ -85,6 +85,7 @@ impl Audit for SecretNonLeakyPathAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/behavioral/sigpipe.rs b/src/audits/behavioral/sigpipe.rs index ec7801c..d07a4ea 100644 --- a/src/audits/behavioral/sigpipe.rs +++ b/src/audits/behavioral/sigpipe.rs @@ -49,6 +49,7 @@ impl Audit for SigpipeAudit { layer: AuditLayer::Behavioral, status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/behavioral/standard_names.rs b/src/audits/behavioral/standard_names.rs index c2c7bce..21447b3 100644 --- a/src/audits/behavioral/standard_names.rs +++ b/src/audits/behavioral/standard_names.rs @@ -21,7 +21,32 @@ use crate::anc_toml::{self, AncConfigLoad}; use crate::audit::Audit; use crate::project::Project; use crate::runner::HelpOutput; -use crate::types::{AuditGroup, AuditLayer, AuditResult, AuditStatus, Confidence}; +use crate::types::{AuditGroup, AuditLayer, AuditResult, AuditStatus, Confidence, MitigationInfo}; + +/// Documentation pointer appended to the audit's Warn evidence so authors of +/// CLIs with social-platform or domain-specific vocabulary discover the +/// `.anc.toml [p6] domain_verbs` opt-in. The pattern doc lives in +/// `docs/solutions/` (a symlink to a separate repo); the path is committed +/// to that repo independently of this crate. +const DOMAIN_VERBS_DOCS_URL: &str = + "docs/solutions/architecture-patterns/anc-toml-domain-verbs-pattern-2026-06-03.md"; + +/// Cap on the number of domain-verb matches listed in the Pass evidence +/// string. Beyond this, the formatter appends `, ...` so text-mode rendering +/// stays tidy without truncating signal needed for the structured +/// `MitigationInfo.domain_match_examples` field. +const DOMAIN_MATCH_EXAMPLES_LIMIT: usize = 5; + +/// Composite result of `audit_standard_names`. `status` is the verdict that +/// drives the scorecard row; `mitigation` is populated only when the verdict +/// was assisted by `.anc.toml [p6] domain_verbs` recognition, giving the +/// scorecard a structured signal that distinguishes a self-declared Pass +/// from an unassisted one. +#[derive(Debug, PartialEq)] +pub(crate) struct StandardNamesResult { + pub status: AuditStatus, + pub mitigation: Option, +} /// Community-standard verbs derived from the spec summary text. Includes both /// CRUD verbs and common meta-commands (`help`, `version`, `init`, etc.) so @@ -95,23 +120,22 @@ const STANDARD_VERBS: &[&str] = &[ "upgrade", // Skill-bundle (P8 alignment) "skill", - // Social / notification platform verbs + // Cross-domain notification / lifecycle verbs (file managers, mail + // clients, notification systems all use these — kept in built-ins). + // Platform-specific verbs (post / repost / unrepost / quote / like / + // unlike / dm) were intentionally removed from this list; they belong + // in per-CLI `.anc.toml [p6] domain_verbs` (see + // `docs/solutions/architecture-patterns/anc-toml-domain-verbs-pattern-2026-06-03.md`). "archive", "block", "bookmark", - "dm", "follow", - "like", "mute", - "post", - "quote", "reply", - "repost", "subscribe", "unarchive", "unblock", "unfollow", - "unlike", "unmute", "unsubscribe", ]; @@ -153,14 +177,20 @@ impl Audit for StandardNamesAudit { // as the primary signal (Warn with the loader's evidence) — a // malformed config is the actionable finding here; the verb check // would only mask it. - let status = match anc_toml::load(&project.path) { - AncConfigLoad::Invalid(msg) => AuditStatus::Warn(msg), + let result = match anc_toml::load(&project.path) { + AncConfigLoad::Invalid(msg) => StandardNamesResult { + status: AuditStatus::Warn(msg), + mitigation: None, + }, other => { let cfg = other.as_config(); let domain_verbs: &[String] = cfg.map(|c| c.p6.domain_verbs.as_slice()).unwrap_or(&[]); match project.help_output() { - None => AuditStatus::Skip("could not probe --help".into()), + None => StandardNamesResult { + status: AuditStatus::Skip("could not probe --help".into()), + mitigation: None, + }, Some(help) => audit_standard_names(help, domain_verbs), } } @@ -171,8 +201,9 @@ impl Audit for StandardNamesAudit { label: self.label().into(), group: self.group(), layer: self.layer(), - status, + status: result.status, confidence: Confidence::Low, + mitigation: result.mitigation, }) } } @@ -185,47 +216,104 @@ impl Audit for StandardNamesAudit { /// platform vocabulary (typically loaded from `.anc.toml`). Recognition is /// case-insensitive on the subcommand name; entries in `domain_verbs` are /// matched verbatim against the lower-cased name. -pub(crate) fn audit_standard_names(help: &HelpOutput, domain_verbs: &[String]) -> AuditStatus { +/// +/// When at least one subcommand is recognized via `domain_verbs` (not via +/// the built-in list), the returned `mitigation` field carries the +/// transparency signal so the scorecard distinguishes a domain-assisted +/// Pass from an unassisted one. `mitigation` is `None` for built-ins-only +/// Pass, for Warn, and for Skip. +pub(crate) fn audit_standard_names( + help: &HelpOutput, + domain_verbs: &[String], +) -> StandardNamesResult { let standard: HashSet<&str> = STANDARD_VERBS.iter().copied().collect(); let domain: HashSet<&str> = domain_verbs.iter().map(String::as_str).collect(); let subs: Vec<&String> = help.subcommands().iter().collect(); if subs.is_empty() { - return AuditStatus::Skip("no subcommands parsed from --help".into()); + return StandardNamesResult { + status: AuditStatus::Skip("no subcommands parsed from --help".into()), + mitigation: None, + }; } let total = subs.len(); - let standard_count = subs - .iter() - .filter(|name| { - let lower = name.to_lowercase(); - standard.contains(lower.as_str()) || domain.contains(lower.as_str()) - }) - .count(); + let mut builtin_matches: Vec<&str> = Vec::new(); + let mut domain_matches: Vec<&str> = Vec::new(); + let mut non_standard: Vec<&str> = Vec::new(); + + for name in subs.iter() { + let lower = name.to_lowercase(); + if standard.contains(lower.as_str()) { + builtin_matches.push(name.as_str()); + } else if domain.contains(lower.as_str()) { + domain_matches.push(name.as_str()); + } else { + non_standard.push(name.as_str()); + } + } - let ratio = standard_count as f32 / total as f32; + let recognized = builtin_matches.len() + domain_matches.len(); + let ratio = recognized as f32 / total as f32; if ratio >= STANDARD_VERB_PASS_RATIO { - AuditStatus::Pass - } else { - let non_standard: Vec<&str> = subs - .iter() - .filter(|name| { - let lower = name.to_lowercase(); - !standard.contains(lower.as_str()) && !domain.contains(lower.as_str()) + let mitigation = if domain_matches.is_empty() { + None + } else { + let examples: Vec = domain_matches + .iter() + .take(DOMAIN_MATCH_EXAMPLES_LIMIT) + .map(|s| (*s).to_string()) + .collect(); + Some(MitigationInfo { + using_domain_verbs: true, + domain_match_count: domain_matches.len(), + domain_match_examples: examples, + builtin_match_count: builtin_matches.len(), + subcommand_total: total, }) - .map(|s| s.as_str()) - .collect(); - AuditStatus::Warn(format!( - "{}/{} subcommand(s) follow standard verb names. Non-standard: {}. \ - MAY-tier — community-standard verbs (get/list/create/update/delete) \ - help agents predict subcommand behavior across CLIs.", - standard_count, - total, - non_standard.join(", ") - )) + }; + StandardNamesResult { + status: AuditStatus::Pass, + mitigation, + } + } else { + StandardNamesResult { + status: AuditStatus::Warn(format!( + "{}/{} subcommand(s) follow standard verb names. Non-standard: {}. \ + MAY-tier — community-standard verbs (get/list/create/update/delete) \ + help agents predict subcommand behavior across CLIs. \ + Per-CLI vocabulary (social, billing, etc.) can opt in via .anc.toml \ + [p6] domain_verbs; see {}.", + recognized, + total, + non_standard.join(", "), + DOMAIN_VERBS_DOCS_URL, + )), + mitigation: None, + } } } +/// Format the Pass-with-mitigation evidence string surfaced on the +/// `p6-standard-names` scorecard row. Public to `crate` because the +/// scorecard view layer composes the prose from the structured +/// `MitigationInfo` carried on the `AuditResult`. The Pass row's evidence +/// names the built-in vs domain split so a downstream reader sees at a +/// glance how much of the verdict depended on the per-CLI opt-in. +pub(crate) fn format_pass_evidence(mitigation: &MitigationInfo) -> String { + let recognized_total = mitigation.builtin_match_count + mitigation.domain_match_count; + let total = mitigation.subcommand_total; + let examples_rendered = if mitigation.domain_match_count <= DOMAIN_MATCH_EXAMPLES_LIMIT { + mitigation.domain_match_examples.join(", ") + } else { + format!("{}, ...", mitigation.domain_match_examples.join(", ")) + }; + format!( + "{}/{} subcommands standard ({} via .anc.toml [p6].domain_verbs: [{}])", + recognized_total, total, mitigation.domain_match_count, examples_rendered, + ) +} + #[cfg(test)] mod tests { use super::*; @@ -261,14 +349,14 @@ Options: -h, --help Show help "#; - const HELP_SOCIAL_VERBS: &str = r#"Usage: x [OPTIONS] + const HELP_CROSS_DOMAIN_VERBS: &str = r#"Usage: notif [OPTIONS] Commands: - post Publish a post - like Like a post - repost Repost a post - bookmark Bookmark a post - follow Follow a user + archive Archive items + bookmark Bookmark items + follow Follow a thread + mute Mute notifications + subscribe Subscribe to a feed Options: -h, --help Show help @@ -277,20 +365,53 @@ Options: const HELP_MIXED_WITH_MENTIONS: &str = r#"Usage: x [OPTIONS] Commands: + archive Archive a post + follow Follow a user + mentions List mentions + +Options: + -h, --help Show help +"#; + + const HELP_SOCIAL_PLATFORM: &str = r#"Usage: x [OPTIONS] + +Commands: + archive Archive a post + bookmark Bookmark a post + follow Follow a user post Publish a post like Like a post - mentions List mentions Options: -h, --help Show help "#; - const HELP_DUP_BUILTIN: &str = r#"Usage: x [OPTIONS] + const HELP_NONSENSE_VERBS: &str = r#"Usage: nonsense [OPTIONS] + +Commands: + yeet Remove with prejudice + bork Repair a thing + blarg Do the blarg + +Options: + -h, --help Show help +"#; + + const HELP_CASE_MISMATCH: &str = r#"Usage: tool [OPTIONS] Commands: post Publish a post - like Like a post - delete Delete a post + +Options: + -h, --help Show help +"#; + + const HELP_DUP_BUILTIN: &str = r#"Usage: tool [OPTIONS] + +Commands: + archive Archive items + delete Delete items + list List items Options: -h, --help Show help @@ -299,15 +420,21 @@ Options: #[test] fn happy_path_standard_verbs() { let help = HelpOutput::from_raw(HELP_STANDARD_VERBS); - assert_eq!(audit_standard_names(&help, &[]), AuditStatus::Pass); + let r = audit_standard_names(&help, &[]); + assert_eq!(r.status, AuditStatus::Pass); + assert!(r.mitigation.is_none()); } #[test] fn warn_non_standard_majority() { let help = HelpOutput::from_raw(HELP_NON_STANDARD); - match audit_standard_names(&help, &[]) { + match audit_standard_names(&help, &[]).status { AuditStatus::Warn(msg) => { assert!(msg.contains("yeet") || msg.contains("bork") || msg.contains("blarg")); + assert!( + msg.contains(DOMAIN_VERBS_DOCS_URL), + "warn evidence must point at docs/solutions opt-in: {msg}" + ); } other => panic!("expected Warn, got {other:?}"), } @@ -316,22 +443,30 @@ Options: #[test] fn skip_no_subcommands() { let help = HelpOutput::from_raw(HELP_NO_SUBCOMMANDS); - match audit_standard_names(&help, &[]) { + match audit_standard_names(&help, &[]).status { AuditStatus::Skip(msg) => assert!(msg.contains("subcommand")), other => panic!("expected Skip, got {other:?}"), } } #[test] - fn expanded_builtin_recognizes_social_verbs() { - let help = HelpOutput::from_raw(HELP_SOCIAL_VERBS); - assert_eq!(audit_standard_names(&help, &[]), AuditStatus::Pass); + fn builtin_recognizes_cross_domain_verbs() { + // After the platform-verb trim, the surviving cross-domain + // additions (archive, bookmark, follow, mute, subscribe and their + // `un-` partners) stay in the built-in list and Pass unaided. + let help = HelpOutput::from_raw(HELP_CROSS_DOMAIN_VERBS); + let r = audit_standard_names(&help, &[]); + assert_eq!(r.status, AuditStatus::Pass); + assert!( + r.mitigation.is_none(), + "cross-domain built-ins must Pass without using domain_verbs" + ); } #[test] fn mentions_unknown_without_domain_verbs() { let help = HelpOutput::from_raw(HELP_MIXED_WITH_MENTIONS); - match audit_standard_names(&help, &[]) { + match audit_standard_names(&help, &[]).status { AuditStatus::Warn(msg) => { assert!( msg.contains("mentions"), @@ -346,7 +481,14 @@ Options: fn mentions_recognized_with_domain_verbs() { let help = HelpOutput::from_raw(HELP_MIXED_WITH_MENTIONS); let domain = vec!["mentions".to_string()]; - assert_eq!(audit_standard_names(&help, &domain), AuditStatus::Pass); + let r = audit_standard_names(&help, &domain); + assert_eq!(r.status, AuditStatus::Pass); + let m = r + .mitigation + .expect("domain-assisted pass populates mitigation"); + assert!(m.using_domain_verbs); + assert_eq!(m.domain_match_count, 1); + assert_eq!(m.domain_match_examples, vec!["mentions".to_string()]); } #[test] @@ -354,19 +496,169 @@ Options: let help = HelpOutput::from_raw(HELP_MIXED_WITH_MENTIONS); // Regression: an empty domain_verbs slice (the loaded-but-empty // case) must behave identically to `Absent`. - assert!(matches!( - audit_standard_names(&help, &[]), - AuditStatus::Warn(_) - )); + let r = audit_standard_names(&help, &[]); + assert!(matches!(r.status, AuditStatus::Warn(_))); + assert!(r.mitigation.is_none()); } #[test] fn domain_verb_duplicating_builtin_is_harmless() { let help = HelpOutput::from_raw(HELP_DUP_BUILTIN); - // `post` is in the built-in list AND in domain_verbs — recognition - // must dedupe via set-membership semantics, not double-count. - let domain = vec!["post".to_string()]; - let status = audit_standard_names(&help, &domain); - assert_eq!(status, AuditStatus::Pass); + // `archive` is in the built-in list AND in domain_verbs — + // recognition must dedupe via set-membership semantics, not + // double-count. The audit must Pass via built-ins, NOT report + // domain_verbs assistance (because `archive` was matched by the + // built-in check first). + let domain = vec!["archive".to_string()]; + let r = audit_standard_names(&help, &domain); + assert_eq!(r.status, AuditStatus::Pass); + assert!( + r.mitigation.is_none(), + "duplicate domain entry must NOT trigger mitigation when built-in already matches: {:?}", + r.mitigation + ); + } + + // ── R5 adversarial coverage (PR #76 follow-up plan) ────────────────── + + #[test] + fn nonsense_domain_verbs_pass_with_transparency_flag() { + // R5(a): a CLI whose subcommands are all nonsense words can Pass + // by declaring those nonsense words in `.anc.toml domain_verbs`. + // The audit Passes (this is the user's risk to take) but the + // mitigation field MUST surface the assistance so the scorecard + // distinguishes a self-declared Pass from an earned one. + let help = HelpOutput::from_raw(HELP_NONSENSE_VERBS); + let domain = vec!["yeet".to_string(), "bork".to_string(), "blarg".to_string()]; + let r = audit_standard_names(&help, &domain); + assert_eq!(r.status, AuditStatus::Pass); + let m = r + .mitigation + .expect("self-declared pass must populate mitigation"); + assert!(m.using_domain_verbs); + assert_eq!(m.domain_match_count, 3); + // Examples preserve encounter order (matches subcommand order + // in --help output). + assert_eq!( + m.domain_match_examples, + vec!["yeet".to_string(), "bork".to_string(), "blarg".to_string()] + ); + } + + #[test] + fn case_mismatch_domain_verbs_does_not_match() { + // R5(b): `domain_verbs` entries are compared verbatim against the + // lowercased subcommand name. `"Post"` (capital P) does NOT match + // `post` (lowercased by the audit). Documented behavior; pinning + // it prevents a silent regression if someone "fixes" the case + // handling without thinking about the contract. + let help = HelpOutput::from_raw(HELP_CASE_MISMATCH); + let domain = vec!["Post".to_string()]; + let r = audit_standard_names(&help, &domain); + // `post` not matched by the case-sensitive domain check; 0/1 + // recognized; the audit Warns. + assert!(matches!(r.status, AuditStatus::Warn(_))); + assert!(r.mitigation.is_none()); + } + + #[test] + fn pass_without_anc_toml_omits_transparency_fields() { + // R5(c): a built-ins-only Pass MUST NOT carry mitigation. Tested + // here as the unit-level contract; the scorecard JSON elision + // (`skip_serializing_if = "Option::is_none"`) is a downstream + // consequence. + let help = HelpOutput::from_raw(HELP_STANDARD_VERBS); + let r = audit_standard_names(&help, &[]); + assert_eq!(r.status, AuditStatus::Pass); + assert!(r.mitigation.is_none()); + } + + #[test] + fn empty_domain_verbs_omits_transparency_fields() { + // R5(d): a `.anc.toml` with `[p6] domain_verbs = []` must behave + // identically to an absent file for transparency purposes — Pass + // (when warranted) carries no mitigation; the audit row is + // byte-identical to the no-config case. + let help = HelpOutput::from_raw(HELP_STANDARD_VERBS); + let domain: Vec = Vec::new(); + let r = audit_standard_names(&help, &domain); + assert_eq!(r.status, AuditStatus::Pass); + assert!(r.mitigation.is_none()); + } + + #[test] + fn social_cli_documented_example_passes() { + // R5(e): the social-CLI example from the docs/solutions pattern + // (xurl-rs-shaped vocabulary) Passes when `.anc.toml` declares the + // platform vocabulary. Mitigation surfaces every platform verb + // that survived the built-in trim; the survival of `archive`, + // `bookmark`, `follow` as built-ins keeps the ratio at Pass + // without help, so the test must assert specifically against the + // platform-only matches. + let help = HelpOutput::from_raw(HELP_SOCIAL_PLATFORM); + let domain = vec![ + "post".to_string(), + "like".to_string(), + "repost".to_string(), + "dm".to_string(), + "quote".to_string(), + ]; + let r = audit_standard_names(&help, &domain); + assert_eq!(r.status, AuditStatus::Pass); + let m = r + .mitigation + .expect("social-CLI Pass via domain_verbs populates mitigation"); + assert!(m.using_domain_verbs); + // HELP_SOCIAL_PLATFORM lists 5 commands: archive, bookmark, follow + // (built-ins) + post, like (domain). domain_match_count is 2. + assert_eq!(m.domain_match_count, 2); + assert_eq!( + m.domain_match_examples, + vec!["post".to_string(), "like".to_string()] + ); + } + + #[test] + fn format_pass_evidence_caps_examples() { + // The Pass evidence string lists at most the first + // DOMAIN_MATCH_EXAMPLES_LIMIT (5) domain matches and appends + // `, ...` when more matches exist. Pins the truncation contract + // so text-mode rendering stays predictable. + let m = MitigationInfo { + using_domain_verbs: true, + domain_match_count: 7, + domain_match_examples: vec![ + "a".to_string(), + "b".to_string(), + "c".to_string(), + "d".to_string(), + "e".to_string(), + ], + builtin_match_count: 3, + subcommand_total: 12, + }; + let evidence = format_pass_evidence(&m); + assert!( + evidence.ends_with("[a, b, c, d, e, ...])"), + "expected truncated example list with trailing ellipsis: {evidence}" + ); + assert!(evidence.starts_with("10/12 subcommands standard (7 via .anc.toml")); + } + + #[test] + fn format_pass_evidence_omits_ellipsis_when_at_limit() { + let m = MitigationInfo { + using_domain_verbs: true, + domain_match_count: 3, + domain_match_examples: vec!["x".to_string(), "y".to_string(), "z".to_string()], + builtin_match_count: 2, + subcommand_total: 7, + }; + let evidence = format_pass_evidence(&m); + assert!( + evidence.ends_with("[x, y, z])"), + "expected non-truncated example list: {evidence}" + ); + assert!(evidence.starts_with("5/7 subcommands standard")); } } diff --git a/src/audits/behavioral/stdin_input.rs b/src/audits/behavioral/stdin_input.rs index fe95f66..f99d3b6 100644 --- a/src/audits/behavioral/stdin_input.rs +++ b/src/audits/behavioral/stdin_input.rs @@ -76,6 +76,7 @@ impl Audit for StdinInputAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/behavioral/structured_exit_codes.rs b/src/audits/behavioral/structured_exit_codes.rs index 01de9e4..4cc1d40 100644 --- a/src/audits/behavioral/structured_exit_codes.rs +++ b/src/audits/behavioral/structured_exit_codes.rs @@ -54,6 +54,7 @@ impl Audit for StructuredExitCodesAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/behavioral/subcommand_examples.rs b/src/audits/behavioral/subcommand_examples.rs index 92a190b..d7ed3fb 100644 --- a/src/audits/behavioral/subcommand_examples.rs +++ b/src/audits/behavioral/subcommand_examples.rs @@ -74,6 +74,7 @@ impl Audit for SubcommandExamplesAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/behavioral/subcommand_operations.rs b/src/audits/behavioral/subcommand_operations.rs index 92df609..d91df4b 100644 --- a/src/audits/behavioral/subcommand_operations.rs +++ b/src/audits/behavioral/subcommand_operations.rs @@ -83,6 +83,7 @@ impl Audit for SubcommandOperationsAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/behavioral/timeout_behavioral.rs b/src/audits/behavioral/timeout_behavioral.rs index 4371a33..f62498a 100644 --- a/src/audits/behavioral/timeout_behavioral.rs +++ b/src/audits/behavioral/timeout_behavioral.rs @@ -64,6 +64,7 @@ impl Audit for TimeoutBehavioralAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/behavioral/verbose_flag.rs b/src/audits/behavioral/verbose_flag.rs index 6479c7a..1854c96 100644 --- a/src/audits/behavioral/verbose_flag.rs +++ b/src/audits/behavioral/verbose_flag.rs @@ -48,6 +48,7 @@ impl Audit for VerboseFlagAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/behavioral/version.rs b/src/audits/behavioral/version.rs index ba3a61d..372a006 100644 --- a/src/audits/behavioral/version.rs +++ b/src/audits/behavioral/version.rs @@ -64,6 +64,7 @@ impl Audit for VersionAudit { layer: AuditLayer::Behavioral, status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/project/agents_md.rs b/src/audits/project/agents_md.rs index bb6102c..8a7c3cd 100644 --- a/src/audits/project/agents_md.rs +++ b/src/audits/project/agents_md.rs @@ -50,6 +50,7 @@ impl Audit for AgentsMdAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/project/bundle_exists.rs b/src/audits/project/bundle_exists.rs index c73858f..7fb0fea 100644 --- a/src/audits/project/bundle_exists.rs +++ b/src/audits/project/bundle_exists.rs @@ -56,6 +56,7 @@ impl Audit for BundleExistsAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/project/completions.rs b/src/audits/project/completions.rs index b7edc38..4d1b605 100644 --- a/src/audits/project/completions.rs +++ b/src/audits/project/completions.rs @@ -65,6 +65,7 @@ impl Audit for CompletionsAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/project/dependencies.rs b/src/audits/project/dependencies.rs index 53ed349..c0531cb 100644 --- a/src/audits/project/dependencies.rs +++ b/src/audits/project/dependencies.rs @@ -76,6 +76,7 @@ impl Audit for DependenciesAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/project/dry_run.rs b/src/audits/project/dry_run.rs index a67e19d..10581b8 100644 --- a/src/audits/project/dry_run.rs +++ b/src/audits/project/dry_run.rs @@ -107,6 +107,7 @@ impl Audit for DryRunAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/project/error_module.rs b/src/audits/project/error_module.rs index 2bcbb66..5f61c63 100644 --- a/src/audits/project/error_module.rs +++ b/src/audits/project/error_module.rs @@ -50,6 +50,7 @@ impl Audit for ErrorModuleAudit { layer: self.layer(), status: AuditStatus::Pass, confidence: Confidence::High, + mitigation: None, }); } } @@ -70,6 +71,7 @@ impl Audit for ErrorModuleAudit { layer: self.layer(), status: AuditStatus::Pass, confidence: Confidence::High, + mitigation: None, }); } } @@ -86,6 +88,7 @@ impl Audit for ErrorModuleAudit { "No dedicated error module found (expected src/error.rs or src/errors.rs)".into(), ), confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/project/non_interactive.rs b/src/audits/project/non_interactive.rs index 42557ff..fa7a6a9 100644 --- a/src/audits/project/non_interactive.rs +++ b/src/audits/project/non_interactive.rs @@ -71,6 +71,7 @@ impl Audit for NonInteractiveSourceAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/project/schema_file.rs b/src/audits/project/schema_file.rs index a1eb52f..59e4703 100644 --- a/src/audits/project/schema_file.rs +++ b/src/audits/project/schema_file.rs @@ -48,6 +48,7 @@ impl Audit for SchemaFileAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/source/python/bare_except.rs b/src/audits/source/python/bare_except.rs index f7b5e02..4cc8d2f 100644 --- a/src/audits/source/python/bare_except.rs +++ b/src/audits/source/python/bare_except.rs @@ -59,6 +59,7 @@ impl Audit for BareExceptAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/source/python/enumerate_valid_set.rs b/src/audits/source/python/enumerate_valid_set.rs index 245c0e9..dbfefc5 100644 --- a/src/audits/source/python/enumerate_valid_set.rs +++ b/src/audits/source/python/enumerate_valid_set.rs @@ -73,6 +73,7 @@ impl Audit for EnumerateValidSetPythonAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/source/python/no_color.rs b/src/audits/source/python/no_color.rs index 5361370..f3ca33c 100644 --- a/src/audits/source/python/no_color.rs +++ b/src/audits/source/python/no_color.rs @@ -73,6 +73,7 @@ impl Audit for NoColorPythonAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/source/python/sigterm.rs b/src/audits/source/python/sigterm.rs index a106016..81efe91 100644 --- a/src/audits/source/python/sigterm.rs +++ b/src/audits/source/python/sigterm.rs @@ -98,6 +98,7 @@ impl Audit for SigtermPythonAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/source/python/sys_exit.rs b/src/audits/source/python/sys_exit.rs index 84bd14d..c245911 100644 --- a/src/audits/source/python/sys_exit.rs +++ b/src/audits/source/python/sys_exit.rs @@ -65,6 +65,7 @@ impl Audit for SysExitAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/source/rust/enumerate_valid_set.rs b/src/audits/source/rust/enumerate_valid_set.rs index 23acaf0..5ff49c6 100644 --- a/src/audits/source/rust/enumerate_valid_set.rs +++ b/src/audits/source/rust/enumerate_valid_set.rs @@ -79,6 +79,7 @@ impl Audit for EnumerateValidSetAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/source/rust/env_flags.rs b/src/audits/source/rust/env_flags.rs index fa84139..f6690c3 100644 --- a/src/audits/source/rust/env_flags.rs +++ b/src/audits/source/rust/env_flags.rs @@ -84,6 +84,7 @@ impl Audit for EnvFlagsAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/source/rust/error_types.rs b/src/audits/source/rust/error_types.rs index 471e06f..d69d74b 100644 --- a/src/audits/source/rust/error_types.rs +++ b/src/audits/source/rust/error_types.rs @@ -69,6 +69,7 @@ impl Audit for ErrorTypesAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/source/rust/exit_codes.rs b/src/audits/source/rust/exit_codes.rs index 115d834..4cf0954 100644 --- a/src/audits/source/rust/exit_codes.rs +++ b/src/audits/source/rust/exit_codes.rs @@ -66,6 +66,7 @@ impl Audit for ExitCodesAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/source/rust/global_flags.rs b/src/audits/source/rust/global_flags.rs index 5f691ad..29b0280 100644 --- a/src/audits/source/rust/global_flags.rs +++ b/src/audits/source/rust/global_flags.rs @@ -86,6 +86,7 @@ impl Audit for GlobalFlagsAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/source/rust/headless_auth.rs b/src/audits/source/rust/headless_auth.rs index a7b02c1..d89f970 100644 --- a/src/audits/source/rust/headless_auth.rs +++ b/src/audits/source/rust/headless_auth.rs @@ -94,6 +94,7 @@ impl Audit for HeadlessAuthAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/source/rust/naked_println.rs b/src/audits/source/rust/naked_println.rs index a4cebcb..f2dabe1 100644 --- a/src/audits/source/rust/naked_println.rs +++ b/src/audits/source/rust/naked_println.rs @@ -76,6 +76,7 @@ impl Audit for NakedPrintlnAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/source/rust/no_color.rs b/src/audits/source/rust/no_color.rs index 80b0483..21a3e8a 100644 --- a/src/audits/source/rust/no_color.rs +++ b/src/audits/source/rust/no_color.rs @@ -72,6 +72,7 @@ impl Audit for NoColorSourceAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/source/rust/no_pager.rs b/src/audits/source/rust/no_pager.rs index 3e0d2ea..42aa664 100644 --- a/src/audits/source/rust/no_pager.rs +++ b/src/audits/source/rust/no_pager.rs @@ -79,6 +79,7 @@ impl Audit for NoPagerAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/source/rust/output_clamping.rs b/src/audits/source/rust/output_clamping.rs index d69676c..1ac7006 100644 --- a/src/audits/source/rust/output_clamping.rs +++ b/src/audits/source/rust/output_clamping.rs @@ -91,6 +91,7 @@ impl Audit for OutputClampingAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/source/rust/output_module.rs b/src/audits/source/rust/output_module.rs index 9d78682..558529f 100644 --- a/src/audits/source/rust/output_module.rs +++ b/src/audits/source/rust/output_module.rs @@ -59,6 +59,7 @@ impl Audit for OutputModuleAudit { layer: self.layer(), status: AuditStatus::Pass, confidence: Confidence::High, + mitigation: None, }); } } @@ -74,6 +75,7 @@ impl Audit for OutputModuleAudit { .into(), ), confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/source/rust/process_exit.rs b/src/audits/source/rust/process_exit.rs index e9622d1..a5ee495 100644 --- a/src/audits/source/rust/process_exit.rs +++ b/src/audits/source/rust/process_exit.rs @@ -64,6 +64,7 @@ impl Audit for ProcessExitAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/source/rust/sigterm.rs b/src/audits/source/rust/sigterm.rs index 27f399b..70ddb57 100644 --- a/src/audits/source/rust/sigterm.rs +++ b/src/audits/source/rust/sigterm.rs @@ -112,6 +112,7 @@ impl Audit for SigtermAudit { layer: self.layer(), status, confidence: Confidence::Medium, + mitigation: None, }) } } diff --git a/src/audits/source/rust/structured_output.rs b/src/audits/source/rust/structured_output.rs index 9478059..0f29749 100644 --- a/src/audits/source/rust/structured_output.rs +++ b/src/audits/source/rust/structured_output.rs @@ -79,6 +79,7 @@ impl Audit for StructuredOutputAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/source/rust/timeout_flag.rs b/src/audits/source/rust/timeout_flag.rs index 76b4f76..2c7ccbf 100644 --- a/src/audits/source/rust/timeout_flag.rs +++ b/src/audits/source/rust/timeout_flag.rs @@ -82,6 +82,7 @@ impl Audit for TimeoutFlagAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/source/rust/try_parse.rs b/src/audits/source/rust/try_parse.rs index 606d6f4..c0c0571 100644 --- a/src/audits/source/rust/try_parse.rs +++ b/src/audits/source/rust/try_parse.rs @@ -65,6 +65,7 @@ impl Audit for TryParseAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/source/rust/tty_detection.rs b/src/audits/source/rust/tty_detection.rs index 1d9e3b2..74c8582 100644 --- a/src/audits/source/rust/tty_detection.rs +++ b/src/audits/source/rust/tty_detection.rs @@ -104,6 +104,7 @@ impl Audit for TtyDetectionAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/audits/source/rust/unwrap.rs b/src/audits/source/rust/unwrap.rs index 759581e..e1ef4c5 100644 --- a/src/audits/source/rust/unwrap.rs +++ b/src/audits/source/rust/unwrap.rs @@ -68,6 +68,7 @@ impl Audit for UnwrapAudit { layer: self.layer(), status, confidence: Confidence::High, + mitigation: None, }) } } diff --git a/src/main.rs b/src/main.rs index 7d45b77..a43f150 100644 --- a/src/main.rs +++ b/src/main.rs @@ -247,6 +247,7 @@ fn run(raw_argv: Vec) -> Result { cat.as_kebab_case() )), confidence: Confidence::High, + mitigation: None, }); continue; } @@ -259,6 +260,7 @@ fn run(raw_argv: Vec) -> Result { layer: audit.layer(), status: AuditStatus::Error(e.to_string()), confidence: Confidence::High, + mitigation: None, }, }; results.push(result); diff --git a/src/principles/matrix.rs b/src/principles/matrix.rs index 9acf016..4711e2e 100644 --- a/src/principles/matrix.rs +++ b/src/principles/matrix.rs @@ -385,6 +385,7 @@ mod tests { layer: AuditLayer::Behavioral, status: AuditStatus::Pass, confidence: Confidence::High, + mitigation: None, }) } fn covers(&self) -> &'static [&'static str] { diff --git a/src/scorecard/audience.rs b/src/scorecard/audience.rs index 362c0e8..0f43655 100644 --- a/src/scorecard/audience.rs +++ b/src/scorecard/audience.rs @@ -148,6 +148,7 @@ mod tests { layer: AuditLayer::Behavioral, status, confidence: Confidence::High, + mitigation: None, } } diff --git a/src/scorecard/mod.rs b/src/scorecard/mod.rs index 0006ad2..41baa79 100644 --- a/src/scorecard/mod.rs +++ b/src/scorecard/mod.rs @@ -23,8 +23,14 @@ use crate::types::{AuditGroup, AuditLayer, AuditResult, AuditStatus}; /// rather than a round-trip to the site), `0.6` (7-status taxonomy: /// `opt_out` + `n_a` added to `status`; matching counters in `summary`; /// `tier` field on each result; one result per requirement-row instead of -/// per-audit_id; antecedent propagation for conditional rows). -pub const SCHEMA_VERSION: &str = "0.7"; +/// per-audit_id; antecedent propagation for conditional rows), `0.7` +/// (unchanged shape over `0.6` per the role-based validators handoff; +/// reserved bump for the JSON-error envelope reframe), `0.8` +/// (`using_domain_verbs` and `domain_match_count` optional fields on +/// each row, populated when `p6-standard-names` Passes via per-CLI +/// `.anc.toml [p6] domain_verbs` recognition; Pass evidence string +/// populated with the built-in / domain ratio). +pub const SCHEMA_VERSION: &str = "0.8"; /// Eligibility floor for the agent-native badge, expressed as an integer /// percent. A score that meets or exceeds this floor qualifies a tool to @@ -392,6 +398,18 @@ pub struct AuditResultView { /// (legacy test fixtures that hand-build a `AuditResult` without the /// fan-out pipeline). pub audit_id: String, + /// Transparency for verdicts assisted by a documented opt-in. Today: + /// `true` when `p6-standard-names` Passed because at least one + /// subcommand was recognized via `.anc.toml [p6] domain_verbs` (not + /// via the built-in `STANDARD_VERBS` list). Absent (`None`, elided + /// from JSON) for every other row. Schema `0.8` addition. + #[serde(skip_serializing_if = "Option::is_none")] + pub using_domain_verbs: Option, + /// Count of subcommands recognized via `domain_verbs` (companion to + /// `using_domain_verbs`). Absent for non-mitigated rows. Schema `0.8` + /// addition. + #[serde(skip_serializing_if = "Option::is_none")] + pub domain_match_count: Option, } impl AuditResultView { @@ -412,7 +430,18 @@ impl AuditResultView { /// requirement row id. pub fn from_row(r: &AuditResult, audit_id: &str) -> Self { let (status, evidence) = match &r.status { - AuditStatus::Pass => ("pass".to_string(), None), + AuditStatus::Pass => { + // When a Pass was assisted by `domain_verbs`, surface the + // formatted ratio + matched names in the row's `evidence` + // field so text-mode rendering and JSON-mode dispatch see + // the same prose. Pass without mitigation keeps the + // existing `evidence: null` shape. + let pass_evidence = r + .mitigation + .as_ref() + .map(crate::audits::behavioral::standard_names::format_pass_evidence); + ("pass".to_string(), pass_evidence) + } AuditStatus::Warn(e) => ("warn".to_string(), Some(e.clone())), AuditStatus::Fail(e) => ("fail".to_string(), Some(e.clone())), AuditStatus::OptOut(e) => ("opt_out".to_string(), Some(e.clone())), @@ -420,6 +449,10 @@ impl AuditResultView { AuditStatus::Skip(e) => ("skip".to_string(), Some(e.clone())), AuditStatus::Error(e) => ("error".to_string(), Some(e.clone())), }; + let (using_domain_verbs, domain_match_count) = match &r.mitigation { + Some(m) => (Some(m.using_domain_verbs), Some(m.domain_match_count)), + None => (None, None), + }; // Serialize AuditGroup / AuditLayer / Confidence via serde_json so // the JSON mirrors the canonical enum spelling (snake_case). let group = serde_json::to_value(r.group) @@ -451,6 +484,8 @@ impl AuditResultView { confidence, tier, audit_id: audit_id.to_string(), + using_domain_verbs, + domain_match_count, } } } @@ -954,6 +989,7 @@ mod tests { layer: AuditLayer::Behavioral, status, confidence: Confidence::High, + mitigation: None, } } @@ -994,7 +1030,7 @@ mod tests { ]; let json = format_json(&results, &[], None, None, fixture_metadata()); let parsed: serde_json::Value = serde_json::from_str(&json).expect("valid JSON"); - assert_eq!(parsed["schema_version"], "0.7"); + assert_eq!(parsed["schema_version"], "0.8"); assert_eq!(parsed["summary"]["total"], 2); assert_eq!(parsed["summary"]["pass"], 1); assert_eq!(parsed["summary"]["fail"], 1); @@ -1213,7 +1249,7 @@ mod tests { let parsed: serde_json::Value = serde_json::from_str(&json).expect("valid JSON"); assert_eq!(parsed["audience"], "agent-optimized"); assert!(parsed["audit_profile"].is_null()); - assert_eq!(parsed["schema_version"], "0.7"); + assert_eq!(parsed["schema_version"], "0.8"); } #[test] @@ -1489,7 +1525,7 @@ mod tests { } // 0.4 + 0.5 additions — every documented sub-key resolves. - assert_eq!(parsed["schema_version"], "0.7"); + assert_eq!(parsed["schema_version"], "0.8"); for path in [ // 0.4 "tool.name", diff --git a/src/types.rs b/src/types.rs index 631b4f6..59c099f 100644 --- a/src/types.rs +++ b/src/types.rs @@ -92,6 +92,49 @@ pub struct AuditResult { /// heuristic audits downgrade. Additive field; consumers feature-detect. #[serde(default)] pub confidence: Confidence, + /// Per-audit transparency carrier: when an audit's Pass depended on a + /// per-CLI mitigation (today: `.anc.toml [p6] domain_verbs` for + /// `p6-standard-names`), the audit populates this so the scorecard + /// distinguishes a self-declared Pass from an unassisted one. `None` for + /// every audit that has no mitigation to declare. Carrier-shaped rather + /// than audit-specific so future audits with similar transparency needs + /// reuse the slot instead of growing parallel fields. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub mitigation: Option, +} + +/// Transparency metadata attached to an `AuditResult` when its verdict +/// depended on a documented opt-in (config-driven recognition, suppression +/// profile, etc.). Distinct from `evidence`, which is prose; `MitigationInfo` +/// is the structured signal a downstream consumer (scorecard renderer, +/// leaderboard) can dispatch on without parsing the evidence string. +/// +/// Current uses: +/// - `p6-standard-names`: when one or more subcommands matched the audit +/// target's `.anc.toml [p6] domain_verbs` list (not the built-in +/// `STANDARD_VERBS`), the audit fills `domain_match_count` and +/// `domain_match_examples`. +#[derive(Debug, Clone, Serialize, PartialEq)] +pub struct MitigationInfo { + /// True iff the verdict was assisted by the named opt-in. Always present + /// when `MitigationInfo` itself is present (the `Some` sentinel of the + /// parent `Option` is the same fact, but consumers find the explicit + /// flag easier to dispatch on). + pub using_domain_verbs: bool, + /// Count of subcommands recognized via `domain_verbs` (not via the + /// built-in standard-verb list). + pub domain_match_count: usize, + /// Up to the first 5 domain-verb matches in encounter order, for + /// display alongside the evidence string. Truncated set; consumers that + /// need every match should re-derive from the CLI's `--help` output. + pub domain_match_examples: Vec, + /// Count of subcommands recognized via the built-in `STANDARD_VERBS` + /// list (not via `domain_verbs`). Combined with `domain_match_count`, + /// this lets a consumer compute the recognized fraction without + /// re-running the audit probe. + pub builtin_match_count: usize, + /// Total subcommand count (denominator of the "recognized" fraction). + pub subcommand_total: usize, } /// A source location where a violation was found. diff --git a/tests/integration.rs b/tests/integration.rs index 9d64a04..8b3ea8d 100644 --- a/tests/integration.rs +++ b/tests/integration.rs @@ -859,7 +859,7 @@ fn test_audit_profile_echoed_in_json_output() { let json_str = String::from_utf8(output).expect("utf8 stdout"); let parsed: serde_json::Value = serde_json::from_str(&json_str).expect("valid JSON"); assert_eq!(parsed["audit_profile"], "human-tui"); - assert_eq!(parsed["schema_version"], "0.7"); + assert_eq!(parsed["schema_version"], "0.8"); } #[test] @@ -1049,7 +1049,7 @@ fn test_scorecard_json_has_stable_top_level_keys() { ); // Fixed enumerations also pin against the renderer contract. - assert_eq!(obj["schema_version"], "0.7"); + assert_eq!(obj["schema_version"], "0.8"); } #[test] diff --git a/tests/scorecard_metadata_security.rs b/tests/scorecard_metadata_security.rs index abcb0dd..525a20e 100644 --- a/tests/scorecard_metadata_security.rs +++ b/tests/scorecard_metadata_security.rs @@ -141,7 +141,7 @@ fn hostile_binary_nonzero_version_exit_yields_null() { ); // The scorecard itself must still emit — version probe failure is not // a scoring failure. - assert_eq!(parsed["schema_version"], "0.7"); + assert_eq!(parsed["schema_version"], "0.8"); assert_eq!(parsed["target"]["kind"], "binary"); } diff --git a/tests/scorecard_schema_v05.rs b/tests/scorecard_schema_v05.rs index dfee447..acfbe93 100644 --- a/tests/scorecard_schema_v05.rs +++ b/tests/scorecard_schema_v05.rs @@ -24,8 +24,8 @@ fn fixture_path(name: &str) -> String { /// precise failure message when a field is missing. fn assert_v05_shape(parsed: &Value) { assert_eq!( - parsed["schema_version"], "0.7", - "schema_version must be 0.7 (per-row emission + 7-status taxonomy)", + parsed["schema_version"], "0.8", + "schema_version must be 0.8 (per-row emission + 7-status taxonomy + domain_verbs transparency)", ); for path in [ @@ -353,8 +353,8 @@ fn rt_schema_id_pins_to_published_schema_version() { let schema = schema_doc(); let id = schema["$id"].as_str().expect("$id is a string"); assert!( - id.contains("scorecard-v0.7"), - "schema $id must pin to the current SCHEMA_VERSION (0.7), got: {id}", + id.contains("scorecard-v0.8"), + "schema $id must pin to the current SCHEMA_VERSION (0.8), got: {id}", ); } diff --git a/tests/standard_names_integration.rs b/tests/standard_names_integration.rs index 7e1f0c1..961b3d1 100644 --- a/tests/standard_names_integration.rs +++ b/tests/standard_names_integration.rs @@ -31,22 +31,22 @@ fn unique_tempdir(label: &str) -> PathBuf { dir } -/// Shell fixture exposing three subcommands (`post`, `like`, `mentions`). -/// `post` + `like` are built-in standard verbs after the social-platform -/// expansion (2/3 = 0.67, below the 0.70 pass threshold); `mentions` is -/// X-specific and must come from `.anc.toml [p6] domain_verbs` to push -/// the ratio across the bar (3/3 = 1.0). `help` is intentionally omitted -/// from the help block — clap always emits it, but including it would -/// add a fourth standard verb and let the fixture pass without exercising -/// the loader at all. +/// Shell fixture exposing three subcommands (`archive`, `follow`, `mentions`). +/// `archive` and `follow` are cross-domain built-in standard verbs that +/// survived the platform-verb trim (2/3 = 0.67, below the 0.70 pass +/// threshold); `mentions` is X-specific and must come from `.anc.toml +/// [p6] domain_verbs` to push the ratio across the bar (3/3 = 1.0). `help` +/// is intentionally omitted from the help block — clap always emits it, +/// but including it would add a fourth standard verb and let the fixture +/// pass without exercising the loader at all. const FIXTURE_SCRIPT: &str = r#"#!/bin/sh case "$1" in --help) cat <<'EOF' Usage: x [OPTIONS] Commands: - post Publish a post - like Like a post + archive Archive a post + follow Follow a user mentions List mentions Options: