Skip to content

datasets: seed the first two agent-trace entries - #72

Merged
emmanuelgjr merged 1 commit into
GenAI-Security-Project:mainfrom
gautamgb:dataset/agent-trace-entries
Sep 17, 2026
Merged

emmanuelgjr merged 1 commit into
GenAI-Security-Project:mainfrom
gautamgb:dataset/agent-trace-entries

Conversation

@gautamgb

Copy link
Copy Markdown
Contributor

Summary

The follow-up promised in #62: the two seed entries and index.csv, now that the schema and validator have merged.

  • DSGAI-TRACE-mcp-declared-effect-flip-stale-autoapproval - a tool re-declares destructiveHint: true after an approval policy has already cached it into an auto-approve class, and the next call runs destructively with no human asked.
  • DSGAI-TRACE-mcp-output-schema-expansion-context-overshare - a tool's output schema widens under a stable server identity, and a schema-agnostic context append carries three unentitled fields into the model context, the user-facing answer, and the observability sink.

Both are derived_from_public_documentation: fully synthetic sessions built around a precondition the MCP specification itself establishes (annotations untrusted from an untrusted server, no version field on a tool definition, tools/list_changed as a SHOULD, outputSchema obligations attaching to whatever schema is current), plus CVE-2025-54136 for the first entry, with the entry stating plainly what the CVE does and does not establish about the non-adversarial mechanism it illustrates.

index.csv is rebuilt against main as it stands now, so it includes #67's entry alongside these two - three rows total.

Verification

$ python validate.py
OK: 3 entries + example.json validated against schema.

$ python build_index.py
Wrote index.csv with 3 entries.

Rebased on the current main tip in a clean worktree (not cherry-picked over a stale base), so there's nothing here that predates #67's entry or the merged schema.

@emmanuelgjr

Copy link
Copy Markdown
Contributor

Thanks @gautamgb, reviewed and verified locally on current main together with #73/#74: validate.py passes (3 entries), the rebuilt index.csv matches yours, and the evidence is scoped honestly. The CVE-2025-54136 note on what it does and doesn't establish is exactly the right register.

One change before merge, a repository-wide attribution policy rather than anything about these entries: please remove affiliation from the contributor object in both entries. Attribution in this repo is a person's name plus, optionally, one ORCID iD. orcid stays; it's a persistent, neutral identifier that helps citation. affiliation is free text, and it's the field through which organizational credit would enter the dataset over time, so we're closing it at the schema level in #75 (after which it fails validation).

So each entry becomes:

"contributor": {
  "name": "Bharti, Gautam",
  "orcid": "0009-0001-4448-1438"
}

Once that's pushed I'll approve and merge.

@gautamgb
gautamgb force-pushed the dataset/agent-trace-entries branch from 5264c9c to 1268394 Compare September 17, 2026 16:59
Per the attribution policy ratified in GenAI-Security-Project#75: name and orcid only.
Two-line change per file, nothing else touched.
@gautamgb
gautamgb force-pushed the dataset/agent-trace-entries branch from 1268394 to 08e3d99 Compare September 17, 2026 17:02

@emmanuelgjr emmanuelgjr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Affiliation dropped, ORCID kept — thanks for the quick turnaround. Re-verified on the rebased head: 3 entries validate, index.csv rebuilds identically, 18 tests pass. Merging.

@emmanuelgjr
emmanuelgjr merged commit 6595d74 into GenAI-Security-Project:main Sep 17, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants