Security skills need a model that will do security analysis

[potiuk]

Follow-up to #1260 and #1261.

What

Establish which LLMAO model, if any, Magpie's security family can actually run against — and track ASF Tooling's plan to provide one that will.

The problem

Most instruction-tuned models refuse prompts that ask them to analyse code for exploitability. "Find how this could be broken into" reads as an attack request to a safety filter, regardless of who is asking or why.

That is not an edge case for us. It is the core loop of the security family — security-issue-triage assessing whether a report is a real vulnerability, security-issue-fix reasoning about what an attacker could reach. A model that declines those prompts cannot run those skills at all, and the failure is silent-ish: a refusal looks like an answer.

ASF Tooling are aware of this and have said they are looking to provide an abliterated model — one with the refusal behaviour removed — precisely so security analysis is possible on the gateway.

Why this needs its own issue

#1261 measures cost. This is a different question: capability. A model can be cheap, fast, and completely unable to do the work. If the security family cannot run on any available LLMAO model, that is a finding worth recording explicitly rather than surfacing as a column of anomalous numbers.

It also interacts with #1264: even once a model will perform the analysis, whether Magpie may send security material through the gateway at all is a separate, unresolved question about the privacy classification.

Done when

• Each available model is characterised for refusal behaviour against the security family's actual prompts, not synthetic ones.
• The result is recorded in organizations/ASF/organization.md alongside the other per-model facts — if a model cannot run the security skills, that belongs in known_limitations.
• The status of Tooling's abliterated model is tracked here, with a pointer once it exists.
• If no model can do this work today, docs/modes.md / the security family docs say so rather than leaving adopters to discover it.

Note

Refusal behaviour is not binary — a model may answer some security prompts and decline others, and may vary run to run. Characterise it with enough repetitions to distinguish "declines" from "declined once".