Skip to content

datasets: seed the first two agent-trace entries #72

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
emmanuelgjr merged 1 commit into from
Sep 17, 2026
Merged

datasets: seed the first two agent-trace entries #72

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
262 changes: 262 additions & 0 deletions ..._toolexchange_traces/entries/DSGAI-TRACE-mcp-declared-effect-flip-stale-autoapproval.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,262 @@
{
"$schema": "../schema.json",
"trace_id": "DSGAI-TRACE-mcp-declared-effect-flip-stale-autoapproval",
"title": "Tool re-declares itself destructive after an agent has already auto-approved it",
"description": "An agent binds a tool whose contract declares destructiveHint false, and an approval policy uses that declaration to place the tool in an auto-approve class. The server later re-publishes the same tool with destructiveHint true, without changing its declared server version. Because the approval decision was made once at bind time and keyed to the tool name, the next call runs a destructive operation under a classification the tool no longer satisfies, and no human approval is ever requested. The trace captures the full sequence including the contract change itself.",
"category": "tool_call",
"disposition": "unintentional_failure",

"dsgai_mapping": ["DSGAI06", "DSGAI05", "DSGAI07"],
"owasp_llm_top10_mapping": ["LLM06:2025", "LLM03:2025"],

"agent": {
"framework": "custom",
"tool_protocol": "mcp",
"topology": "single agent with an orchestrator-side approval policy over remote tools",
"autonomy": "human_approves_flagged_actions"
},

"provenance": {
"tier": "derived_from_public_documentation",
"collection_method": "Constructed as a minimal agent session around a condition the Model Context Protocol permits by construction: a tool's annotations can change after a client has bound and classified the tool, and the tool definition carries no version a client could compare. The agent, orchestrator, approval policy, and every payload are synthetic.",
"evidence": [
{
"type": "url",
"citation": "https://modelcontextprotocol.io/specification/2025-06-18/schema",
"locator": "ToolAnnotations > destructiveHint",
"supports": "destructiveHint is a server-declared, optional boolean: 'If true, the tool may perform destructive updates to its environment. If false, the tool performs only additive updates. Default: true.' It is a property of the tool's own declared annotations, not of any external check, so re-declaring it is entirely within what the server is allowed to do at any time."
},
{
"type": "url",
"citation": "https://modelcontextprotocol.io/specification/2025-06-18/server/tools",
"locator": "Data Types > Tool, trust and safety warning",
"supports": "The specification conditions trust on the server, not the annotation: 'clients MUST consider tool annotations to be untrusted unless they come from trusted servers.' A server a client has already bound and classified is, by construction, one the client currently trusts - so the warning does not forbid an auto-approve policy keyed to a trusted server's annotations. What it does not address is whether that trust should survive the server changing the annotation later, which is the condition this trace models: the initial classification is not re-evaluated when the value it was based on changes."
},
{
"type": "url",
"citation": "https://modelcontextprotocol.io/specification/2025-06-18/server/tools",
"locator": "Capabilities; List Changed Notification; Data Types > Tool",
"supports": "A tool definition is name, title, description, inputSchema, outputSchema and annotations, with no version field, and it may change during a session. The only change signal is notifications/tools/list_changed, which servers that declared the listChanged capability SHOULD send, so a client holding a binding made at an earlier tools/list has neither a guaranteed notification nor a version to compare."
},
{
"type": "cve",
"citation": "CVE-2025-54136",
"locator": "NVD description; advisory GHSA-24mc-g4xr-4395",
"supports": "The same failure shape reached shipped software one layer down: an MCP entry a collaborator had already accepted could be swapped for a different command 'without triggering any warning or re-prompt' (Cursor 1.2.4 and below, fixed in 1.3). The CVE concerns the server command in an approved configuration rather than a tool annotation, so what it establishes is narrower than this trace: a once-granted approval outliving a change to what was approved is observed client behaviour, not only a protocol possibility."
}
]
},

"sanitization": {
"attestation": true,
"techniques": ["not_applicable_fully_synthetic", "name_generalization", "shape_only_payloads"],
"notes": "No real server, tool, operator, or tenant is named. The trace is fully synthetic and uses a generic server identity on purpose: attributing a failure mechanism to a named third party is a disclosure act rather than a dataset contribution. The only vendor named in this entry is the one in the cited CVE, which is already public."
},

"spans": [
{
"span_id": "s1",
"parent_span_id": null,
"t_offset_ms": 0,
"actor": "orchestrator",
"actor_id": "orchestrator://session-1",
"operation": "tool.discover",
"summary": "Client lists tools from a remote server and reads the declared effect annotations.",
"payload": {
"server": "tool://server-a",
"tool": "remove_from_roster",
"declared_server_version": "<synthetic:semver>",
"annotations": { "readOnlyHint": false, "destructiveHint": false, "idempotentHint": false },
"input_schema_digest": "<shape:sha256-truncated,len=12>"
},
"data_classes": ["tool_schema"],
"sensitivity": "none"
},
{
"span_id": "s2",
"parent_span_id": "s1",
"t_offset_ms": 40,
"actor": "policy_engine",
"actor_id": "policy://approval",
"operation": "policy.decision",
"summary": "Approval policy reads destructiveHint false and places the tool in the auto-approve class. The decision is cached against the tool name.",
"payload": {
"rule": "write tools may auto-approve when annotations.destructiveHint is not true",
"decision": "auto_approve",
"decision_key": "server-a/remove_from_roster",
"cache_ttl": "session",
"revalidate_on_contract_change": false
},
"data_classes": ["tool_schema"],
"sensitivity": "none",
"finding": {
"dsgai_id": "DSGAI07",
"note": "The authorization decision is derived from a mutable, self-declared attribute of the counterparty and is then cached with no binding to the contract it was derived from. Nothing in the record ties the decision to the specific contract that justified it.",
"severity": "Medium"
}
},
{
"span_id": "s3",
"parent_span_id": "s2",
"t_offset_ms": 900,
"actor": "agent",
"actor_id": "agent://roster-assistant",
"operation": "tool.call",
"summary": "First call. Runs without a human approval prompt, correctly, under the contract as declared at bind time.",
"payload": {
"tool": "server-a/remove_from_roster",
"arguments": { "roster_id": "<synthetic:uuid>", "member_id": "<synthetic:uuid>", "dry_run": true }
},
"data_classes": ["tool_schema"],
"sensitivity": "low"
},
{
"span_id": "s4",
"parent_span_id": "s3",
"t_offset_ms": 1480,
"actor": "tool",
"actor_id": "tool://server-a/remove_from_roster",
"operation": "tool.result",
"summary": "Tool returns a preview of the change. No data is removed.",
"payload": { "status": "ok", "would_remove": 1, "removed": 0 },
"data_classes": ["tool_output"],
"sensitivity": "low"
},
{
"span_id": "s5",
"parent_span_id": null,
"t_offset_ms": 86400000,
"actor": "external_service",
"actor_id": "tool://server-a",
"operation": "contract.change",
"summary": "Server re-publishes the same tool with destructiveHint true. The declared server version is unchanged, so nothing downstream of a version check observes the change.",
"payload": {
"tool": "server-a/remove_from_roster",
"path": "annotations.destructiveHint",
"change_kind": "annotation-flip-to-destructive",
"before": false,
"after": true,
"declared_server_version_before": "<synthetic:semver>",
"declared_server_version_after": "<synthetic:semver>",
"version_delta": "same"
},
"data_classes": ["tool_schema"],
"sensitivity": "none",
"finding": {
"dsgai_id": "DSGAI05",
"note": "The counterparty's declared effect changed while every integrity signal available to the client - declared version, tool name, endpoint - held steady. A client that validates on version alone cannot detect this transition.",
"severity": "High"
}
},
{
"span_id": "s6",
"parent_span_id": null,
"t_offset_ms": 90000000,
"actor": "orchestrator",
"actor_id": "orchestrator://session-2",
"operation": "tool.bind",
"summary": "A later session resumes from the cached binding. No re-discovery is performed, so the new contract is never fetched.",
"payload": {
"bound_from": "cache",
"decision_key": "server-a/remove_from_roster",
"contract_refetched": false
},
"data_classes": ["tool_schema"],
"sensitivity": "none",
"finding": {
"dsgai_id": "DSGAI06",
"note": "The binding is keyed by tool name, so it survives a change to everything the name refers to. This is the step that converts a supplier-side change into a client-side authorization failure.",
"severity": "High"
}
},
{
"span_id": "s7",
"parent_span_id": "s6",
"t_offset_ms": 90000600,
"actor": "agent",
"actor_id": "agent://roster-assistant",
"operation": "tool.call",
"summary": "Second call, this time for real. The tool now declares itself destructive; the agent does not know that.",
"payload": {
"tool": "server-a/remove_from_roster",
"arguments": { "roster_id": "<synthetic:uuid>", "member_id": "<synthetic:uuid>", "dry_run": false }
},
"data_classes": ["tool_schema"],
"sensitivity": "moderate"
},
{
"span_id": "s8",
"parent_span_id": "s7",
"t_offset_ms": 90000610,
"actor": "policy_engine",
"actor_id": "policy://approval",
"operation": "policy.decision",
"summary": "Policy hits the cached auto-approve decision from s2. No approval request is emitted to the human approver.",
"payload": {
"decision": "auto_approve",
"decision_source": "cache",
"decision_derived_from_contract": "s1",
"contract_in_effect": "s5",
"approval_requested": false
},
"data_classes": ["tool_schema"],
"sensitivity": "high",
"finding": {
"dsgai_id": "DSGAI06",
"note": "The auto-approve class was earned by a contract that is no longer in force. The gap between decision_derived_from_contract and contract_in_effect is the whole defect, and it is invisible to every party at runtime.",
"severity": "High"
}
},
{
"span_id": "s9",
"parent_span_id": "s7",
"t_offset_ms": 90001250,
"actor": "tool",
"actor_id": "tool://server-a/remove_from_roster",
"operation": "tool.result",
"summary": "Destructive operation executes. Records are removed with no human in the loop.",
"payload": { "status": "ok", "removed": 1, "reversible": false },
"data_classes": ["tool_output"],
"sensitivity": "high"
},
{
"span_id": "s10",
"parent_span_id": "s9",
"t_offset_ms": 90001300,
"actor": "orchestrator",
"actor_id": "orchestrator://session-2",
"operation": "context.append",
"summary": "Result is appended to the model context and summarized back to the user as a completed action.",
"payload": { "appended_bytes": "<shape:int,approx=180>", "presented_as": "completed action" },
"data_classes": ["tool_output"],
"sensitivity": "low"
}
],

"security_observations": [
"An approval decision derived from a self-declared, mutable attribute of a third party inherits that attribute's mutability, but caching hides the inheritance.",
"A declared version is not an integrity signal for a tool contract. The protocol puts no version on a tool definition at all, so a client has nothing to compare, and the one change signal it defines is a SHOULD from servers that opted into announcing it.",
"Name-keyed tool bindings are the failure amplifier: they let a supplier-side change silently inherit a client-side authorization grant.",
"The failure is fully invisible in the trace as most systems record it. Without the contract.change span at s5, spans s6 through s9 look like a correct, policy-compliant run.",
"No adversary is required. This flow is what ordinary supplier maintenance does to a client that trusts declared effects."
],

"mitigations": [
"Bind approval decisions to a contract digest, not a tool name. Re-derive the decision whenever the digest changes.",
"Re-fetch and re-diff tool contracts at session start rather than restoring a cached binding, and treat any change to declared effect annotations as an approval-invalidating event.",
"Escalate to human approval on the transition itself, independent of the destination state: a tool that becomes destructive is a stronger signal than one that always was.",
"Do not treat a declared server version as evidence that a contract is unchanged; compare the contract.",
"Record, in the action log, which contract version justified each authorization decision, so the gap at s8 is auditable after the fact."
],

"contributor": {
"name": "Bharti, Gautam",
"orcid": "0009-0001-4448-1438"
},

"date_added": "2026-08-07",

"tags": ["mcp", "tool-contract-drift", "declared-effect", "approval-bypass", "excessive-agency", "supply-chain"],

"notes": "MITRE ATLAS mapping is deliberately omitted. Repository issues #59 and #60 show ATLAS identifiers and their names have both moved across releases, and asserting an unverified pair here would add to that debt. The schema supports a release-pinned mapping; this entry should gain one once the repository settles on a pinned release. This entry also does not claim intent on the part of any tool supplier: a contract diff is an observation about declared effect, not a safety verdict."
}
Loading
Loading