Skip to content

Reconcile LLMAO hosting: "local-hosted" vs rented third-party GPU #1264

Description

@potiuk

Follow-up to #1260.

What

Establish where LLMAO's models actually run, and revisit the privacy classification that depends on the answer.

Why this matters

#1226 carved llm.apache.org out of the *.apache.org default-approval in the approved-LLM gate. The whole rationale was the hardware:

LLMAO is NOT default-approved for foundation private data: it serves from rented third-party GPU hardware and pilot traffic is visible to llmao admins.

organizations/ASF/organization.md encodes that as privacy_class: project-internal — public and project-internal material only, no credentials, no embargoed work.

ASF Tooling have since described all three models as local-hosted.

Those two statements may describe the same arrangement in different words — self-hosted models, as opposed to calling a vendor API, running on rented boxes. Or the hosting may have moved. From outside they are indistinguishable, and the difference decides whether the carve-out still has a basis.

Why it can't be left ambiguous

The carve-out is what stops <private-list> and <security-list> content — including embargoed CVE detail — reaching the gateway without an adopter explicitly declaring it. If the premise has changed, the classification should change deliberately and be re-recorded. If it has not, the wording in our docs should stop inviting the question.

Either outcome is fine. Leaving a security control resting on an ambiguity is not.

Questions to put to Tooling

  1. Where do the three models physically run — ASF-controlled hardware, or rented GPU capacity from a third party?
  2. Who can see pilot traffic — prompts, completions, logs — and is that access recorded anywhere an adopter can read?
  3. Is there a retention policy for prompts and completions?
  4. Is the intended end state different from the pilot's, and on what timeline?

Done when

  • The hosting arrangement is stated plainly, sourced, and dated.
  • privacy_class in organizations/ASF/organization.md is either confirmed or changed, with the reasoning recorded.
  • tools/privacy-llm/models.md § Carve-outs from the *.apache.org rule matches whatever is concluded — the carve-out is removed, kept, or kept with a corrected rationale.
  • If the classification loosens, the checker test cases in tools/privacy-llm/checker move with it.

Related

Blocks any decision about running the security family against LLMAO (#1263) — a model that will do security analysis is necessary but not sufficient if we may not send it the material.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    family:securitysecurity-* skillskind:policyRule changes (eligibility, thresholds, behaviour switches)questionFurther information is requested

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions