Skip to content

Skill lifecycle hooks missing from the trace mappings, acs.md, and the Trace pillar docs #58

Description

@rocklambros

The three skill lifecycle hooks (skillRegister, skillLoad, skillUnload) are present in hooks.md, in the specification.md taxonomy table, and as schemas under specification/v0.1.0/hooks/. They are absent from six other surfaces that restate the hook set, two of which are normative.

Found while reviewing #20, whose FAQ correctly states 19. The suite is green on all of this (198 passed).

Normative gaps

specification/v0.1.0/trace/otel-mapping.json and specification/v0.1.0/trace/ocsf-mapping.json each enumerate 16 steps/* keys and contain none of the three skill hooks.

$ python3 -c "import re;print(len(set(re.findall(r'\"(steps/[A-Za-z]+)\"', open('specification/v0.1.0/trace/otel-mapping.json').read()))))"
16

Both files apply to ACS-Trace, and otel-mapping.json states that "v0.1.0 conformance requires that a deployment emitting OTel for the Trace pillar uses these span names and required attributes verbatim." conformance.md:19 says the skill hooks SHOULD be implemented. So a deployment that implements skillRegister and claims ACS-Trace has no normative span name or class_uid to emit. Two conformant implementations can emit incompatible telemetry for the same hook and nothing adjudicates.

The skillRegister case is the one worth caring about, because a denied skillRegister is a malicious skill definition blocked at the vetting gate, which is exactly the event a SIEM should receive.

Documentation gaps

  • docs/acs.md:11 says "16 native lifecycle hooks" and enumerates 16 names. grep -c "skillRegister\|skillLoad\|skillUnload" docs/acs.md returns 0. This is the ACS overview page, so it is where a reader learns what ACS-Core is, and it currently tells them skill hooks are not part of v0.1.0.
  • docs/spec/trace/events.md has zero occurrences of "skill". Its OTel span table and OCSF class table both stop at subagentStart/subagentStop.
  • docs/spec/trace/extend_ocsf.md has zero occurrences of "skill".

hooks.md:3 says 19. So the published site currently states both 16 and 19 in prominent places.

Why CI does not catch it

tests/test_hook_taxonomy.py derives the hook set from specification/v0.1.0/hooks/*.json and compares it against two files:

SPECIFICATION = ROOT / "docs" / "spec" / "instrument" / "specification.md"
HOOKS_PAGE = ROOT / "docs" / "spec" / "instrument" / "hooks.md"

Those are the two files that are already correct. The guard's premise is that the hook set is stated in three places. It is stated in at least nine, and four of the unguarded ones are wrong today.

Suggested fix

  1. Add the three hooks to otel-mapping.json and ocsf-mapping.json with span names and class_uids consistent with the existing 16.
  2. Update docs/acs.md:11, docs/spec/trace/events.md, and docs/spec/trace/extend_ocsf.md.
  3. Widen the guard so it derives the restating surfaces rather than hardcoding two paths. Matching every steps/<name> occurrence across docs/ and specification/v0.1.0/trace/ would have caught all six.

Item 3 matters more than the other two. Without it the next hook addition reopens the same gap, which is what happened here.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions