Skip to content

Ratify: definition of best, verified-core frameworks, and framework-addition order (T-STRAT01) #99

Description

@emmanuelgjr

T-STRAT01 compared this crosswalk with SCF, CSA AICM, MITRE ATLAS, HITRUST AI, NIST OLIR, NIST AI RMF crosswalks,
the OWASP AI Exchange and Google SAIF / CoSAI. Every statement about another project is cited, and sources
were accessed on 2026-09-14. The analysis is in docs/COMPETITIVE_ANALYSIS.md (PR #98).

The analysis proposes a definition of best, a verified-core list and an order for adding frameworks. None of
these is decided. Each item below needs a maintainer choice. Where a target is a value judgement, the analysis
leaves it as TODO(maintainer) rather than proposing a number.

Findings that bear on these decisions

  • OLIR is no longer empty for OWASP material. OLIR entry 198 maps the OWASP LLM Top 10 (2025) to CSF 2.0
    with 169 typed rows. It is Final, posted 2026-05-08, by an independent non-owner developer. Entry 229 maps
    OWASP AISVS 1.0 to SP 800-53 Rev. 5.2.0. docs/OLIR_SUBMISSION.md said no competing GenAI crosswalk
    appears there; the same PR corrects that sentence.
  • Review and typing. 0 of 3,497 rows on main have a named reviewer or a relationship value.
  • Strength scale. NIST IR 8278A Rev. 1 defines Strength of Relationship as an integer 0–10. The schema
    maps a high/medium/low/unreviewed enum to it.
  • Direction. The exporter code and docs/OLIR_SUBMISSION.md treat the framework control as the Focal
    Document Element. The exporter header and the data/schema.json description treat it as the Reference
    Document Element. Which reading applies decides whether subset-of means what docs/SCHEMA_V2_MIGRATION.md
    says it means.
  • ATLAS identifiers. After fix(atlas): correct eleven MITRE ATLAS technique citations #92, 56 of 117 ATLAS rows carry exactly the 2026.08 name for their ID; 56 rows
    name a different technique or a name never used by ATLAS (for example AML.T0035 labelled "Exfiltrate via ML
    Inference API"), and 8 rows cite IDs absent from 2026.08. The registry is pinned to ATLAS "4.0". Details: MITRE ATLAS: 81 of 117 mapping rows pair an id with another technique's name; 6 ids do not exist (checked against ATLAS 2026.08) #93.

Decision 1: adopt the definition-of-best criteria

Tick one option per criterion. If you pick "adopt with change", say what changes in a comment.

  • B1 % of verified-core rows with a named reviewer and date:
    • adopt
    • adopt with change
    • reject
  • B2 % of verified-core rows with relationship, rationale type and rationale:
    • adopt
    • adopt with change
    • reject
  • B3 vocabulary test aligned to IR 8278A Rev. 1:
    • adopt
    • adopt with change
    • reject
  • B4 identifier integrity against the pinned upstream release:
    • adopt
    • adopt with change
    • reject
  • B5 % of rows with framework_version:
    • adopt
    • adopt with change
    • reject
  • B6 % of frameworks freshness-checked within the SLA window:
    • adopt
    • adopt with change
    • reject
  • B7 days from upstream release to verify / re-map:
    • adopt
    • adopt with change
    • reject
  • B8 reviewed mapping sets accepted in OLIR or by the framework owner:
    • adopt
    • adopt with change
    • reject
  • B9 % of verified-core rows with a confirmed incident failure:
    • adopt
    • adopt with change
    • reject
  • B10 OSCAL exports validate against the NIST schema in CI:
    • adopt
    • adopt with change
    • reject
  • B11 named reviewers with merge rights per verified-core framework:
    • adopt
    • adopt with change
    • reject
  • B12 published npm version equals repository version:
    • adopt
    • adopt with change
    • reject

Decision 2: set the value-judgement targets

Fill in or tick "defer". The analysis proposes no numbers.

  • B1 target: ___ % of verified-core rows reviewed by ___ (date)
    • defer
  • B2 target: ___ % of verified-core rows typed by ___ (date)
    • defer
  • B4 name-match target: ___ %
    • defer (ID resolution itself is proposed at 100%)
  • B5 target: ___ % of rows version-pinned
    • defer
  • B6 and B7: verify within ___ days; re-map within ___ days (also fills docs/FRESHNESS_SLA.md)
    • defer
  • B8 target: ___ accepted sets by ___
    • defer
  • B9 target: ___ %
    • defer
  • B11 target: ___ reviewers per framework
    • defer

Decision 3: verified-core framework list

Tick include or exclude for each.

Decision 4: what "verified" requires

  • One named reviewer per row
  • Two named reviewers per row
  • Named reviewer plus owner-community review (for example, ATLAS maintainers or an ISO liaison) where
    reachable
  • Other: ______

Decision 5: OLIR plan now that entries 198 and 229 exist

  • Keep CSF 2.0 as the first focal document and reconcile against entry 198 before filing
  • Contact the developer of entry 198 about a joint or superseding OWASP-owned submission
  • Choose a different first focal document
    • AI RMF 1.0
    • SP 800-53 Rev. 5.2.0
  • Defer OLIR until the verified core is reviewed

Decision 6: strength of relationship

  • Add an integer 0–10 strength field per IR 8278A Rev. 1 and keep confidence as review state
  • Replace the confidence enum with the 0–10 integer
  • Keep the enum and export Strength of Relationship as "N/A"
  • Other: ______

Decision 7: relationship direction

  • The framework control is the Focal Document Element and the OWASP entry is the Reference Document
    Element, as in the exporter code. Fix the schema description and exporter header, and restate the
    subset-of / superset-of definitions.
  • The OWASP entry is the Focal Document Element. Fix the exporter code and docs/OLIR_SUBMISSION.md.
  • Other: ______

Decision 8: order for adding the dangling frameworks (#19)

Proposed order: NIST AI 600-1, SP 800-53 Rev. 5.2.0, MITRE ATT&CK, ISO/IEC 27701:2025, CycloneDX (ECMA-424),
BSIMM16, COBIT 2019.

  • Accept the order
  • Accept with changes (comment)
  • Declare out of scope
    • BSIMM
    • COBIT
    • CycloneDX (reference as tooling only)

Decision 9: ATLAS identifier mismatch (tracked in #93)

Correcting these rows means choosing the technique each row should cite. That is SME work (C4).

  • SME re-map against ATLAS 2026.08 as part of the verified-core pass
  • Re-sync the registry and add a mechanical ID/name check to validate.js first (no mapping judgement)
  • Defer

Nothing in this issue changes data. Once the decisions are recorded, the ratified sections of
docs/COMPETITIVE_ANALYSIS.md lose their DRAFT markers in a follow-up PR.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions