diff --git a/.github/memory/2026-07-rebrand/research/content-proposals/11-sheet-principles.md b/.github/memory/2026-07-rebrand/research/content-proposals/11-sheet-principles.md index caa82347..a7a91e82 100644 --- a/.github/memory/2026-07-rebrand/research/content-proposals/11-sheet-principles.md +++ b/.github/memory/2026-07-rebrand/research/content-proposals/11-sheet-principles.md @@ -2,15 +2,17 @@ PROPOSAL — for owner review (v2, frontier revision) # Principles copy sheet — The 9 Responsible AI Principles -This version replaces the earlier annotation sheet with the actual proposed content, per principle: frontmatter (title, description, commitment, failure_modes, controls, related_links) followed by the body prose, mapping 1:1 to `src/content/principles/NN.md`. Edit in place. +This version replaces the earlier annotation sheet with the actual proposed content, per principle: a field table (title, tagline, commitment, failure modes, controls, related links) followed by the body prose, mapping 1:1 to the frontmatter and body of `src/content/principles/NN.md`. Edit in place. ## Cross-cutting decisions (ratified in review) - **Persona:** the principles address teams that build systems with AI models: deployers, integrators and fine-tuners. Each entry stays readable by model builders; where the builder obligation differs, the entry states it. - "machine learning systems" wording replaced with "AI systems" throughout, including commitments. -- `description` no longer duplicates the commitment. -- Monitoring lives in 06 (evaluation continued in production); auditability lives in 04 (renamed Reproducibility & Provenance). No tenth principle. +- Titles standardised as X by Y (goal by mechanism); each commitment closes by naming its title (enabling **X by Y**). +- `description` no longer duplicates the commitment; the table's Tagline row maps to the `description` field. +- Monitoring lives in 06 (Evaluation by Observability); auditability lives in 04 (Provenance by Reproducibility). No tenth principle. - Boundary between 02 and 09 written into both bodies: 02 asks who carries the errors, 09 asks what the system is pursuing. +- Failure modes are phrased as the building team's bad practice, not as attacker events or neutral phenomena. - No em-dashes anywhere. ## Proposed index intro (persona statement, for /principles) @@ -19,32 +21,24 @@ This version replaces the earlier annotation sheet with the actual proposed cont --- -## P01 — Human Augmentation - -```yaml -title: Human Augmentation -description: Irrespective of how many levels of abstractions are introduced through AI systems, the impact is and will always continue to be human. -commitment: We commit to assess the consequences of incorrect outputs and automated actions and, to design systems with human oversight to ensure aligned and safe outcomes. -failure_modes: - - Automation of decisions the AI Act treats as high risk or prohibits - - Agents acting beyond their mandate - - Review in name only -controls: - - Impact assessment before automation - - Approval gates for consequential actions - - Bounded autonomy with escalation paths -related_links: - - EU AI Act, Article 14 (human oversight) - - "AI-RFX: operational process design" -``` - -Irrespective of how many levels of abstractions are introduced through AI systems, the impact is and will always continue to be human. AI systems should be developed to augment human cognition and capability as a whole. Human oversight must be enabled, and where reasonable, enabling human-in-the-loop to drive changes to high risk outcomes. This is a requirement now codified in Article 14 of the EU AI Act, which [the institute contributed to through its policy work](/initiatives). +## P01 — Human Augmentation by Oversight + +| Field | Content | +| :---- | :---- | +| Title | Human Augmentation by Oversight | +| Tagline | Irrespective of how many levels of abstractions are introduced through AI systems, the impact is and will always continue to be human. | +| Commitment | We commit to assess the consequences of incorrect outputs and automated actions and to design systems with human oversight to ensure aligned and safe outcomes, enabling **Human Augmentation by Oversight**. | +| Failure modes | Automation of decisions the AI Act treats as high risk or prohibits
Agents acting beyond their mandate
Review in name only | +| Practical controls | Impact assessment before automation
Approval gates for consequential actions
Bounded autonomy with escalation paths | +| Related links | EU AI Act, Article 14 (human oversight)
AI-RFX: operational process design | + +Irrespective of how many levels of abstractions are introduced through AI systems, the impact is and will always continue to be human. AI systems should be developed to augment human cognition and capability as a whole. Human oversight must be enabled, and where reasonable, with human-in-the-loop established to drive remediation to predictions in contexts of high or unacceptable risk outcomes. This is a requirement now codified in Article 14 of the EU AI Act, which [the institute contributed to through its policy work](/initiatives/). The AI Act also draws the automation line in law: Article 5 prohibits some practices outright, such as social scoring and manipulative systems, while the high-risk uses of Annex III, including justice, healthcare and critical infrastructure, may only operate under the human oversight of Article 14. In these domains a single wrong prediction can carry generational impact, so the level of automation must follow the consequence, not the capability. ### Where it fails -Oversight designed for single predictions does not transfer to modern AI & Agentic systems. An agent can take hundreds of actions in one task, so reviewing each one is impossible and in some contexts reviewing none may be negligent. +Oversight designed for single ML models does not transfer to modern AI & Agentic systems. An agent can take hundreds of actions in one task, so reviewing each one is impossible; however, in some contexts reviewing none may be negligent. - Automation that displaces meaningful human judgement. - Agents that operate outside the mandate their operators intended. @@ -58,185 +52,250 @@ Controls should keep accountable people close to consequential decisions and sca - Bound what a system may do alone and gate consequential actions on approval. - Measure outcomes for the people affected, and keep escalation paths that work. -## P02 — Bias Evaluation -[PROPOSED renames, pick one: 1) Bias Calibration (recommended — matches the description's "identified and where required, calibrated" and goes beyond only evaluating); 2) Fairness by Calibration (matches the "X by Y" house pattern of P03/P07); 3) Bias & Fairness (plainest, most searchable); 4) Equitable Impact (names the outcome; pairs cleanly with the 02/09 boundary "who carries the errors").] - -```yaml -title: Bias Evaluation -description: All models have inherent bias; despite this, some can be useful, but some can be harmful. This bias must be identified and where required, calibrated. -commitment: We commit to continuously develop processes that allow us to understand, document and monitor bias in the models we use and across the systems we build on them. -failure_modes: - - Inherited model bias propagated across the system - - Application data and prompts amplifying skew - - Fine-tuning shifting behaviour unnoticed -controls: - - Disaggregated behavioural evaluations - - Review of provider model documentation - - Re-evaluation after fine-tuning and updates -related_links: - - XAI Framework - - "AI-RFX: data & model assessment" -``` - -Bias evaluation should examine who benefits, who carries risk, and where a system's errors concentrate. In systems built on general-purpose models the bias has two layers: 1) what the model inherited from training data that its deployer cannot inspect, and 2) what the application introduces through its own data, prompts, retrieval sources and thresholds. Furthermore, fine-tuning amplifies both layers, as it rewrites the model's behaviour and makes the result the fine-tuner's responsibility. - -This principle asks who carries the skewed impact of the AI system errors. Teams must combine quantitative tests with domain knowledge and affected-community input, then re-evaluate as data, model versions and deployment conditions change. +## P02 — Fairness by Calibration + +| Field | Content | +| :---- | :---- | +| Title | Fairness by Calibration | +| Tagline | All models carry inherent bias; some may be useful, but some can be harmful. This bias must be identified and where relevant, calibrated. | +| Commitment | We commit to introduce processes proportionate to risk that mitigate unintended discrimination across groups in our production AI systems, enabling **Fairness by Calibration**. | +| Failure modes | Inherited model bias propagated across the system
Application data and prompts amplifying skew
Fine-tuning shifting behaviour unnoticed | +| Practical controls | Disaggregated behavioural evaluations
Review of provider model documentation
Re-evaluation after fine-tuning and updates | +| Related links | XAI Framework
AI-RFX: data & model assessment | + +Human and model bias is present throughout the end-to-end AI system lifecycle; bias evaluation supports understanding who benefits, who carries risk, and where a system's errors concentrate. In systems built on general-purpose models the bias has two layers: 1) what the model inherited from training data that its deployer cannot inspect, and 2) what the application introduces through its own data, prompts, retrieval sources and thresholds. + +Furthermore, fine-tuning amplifies both layers, as it rewrites the model's behaviour and makes the result the fine-tuner's responsibility. This is why it is important to ensure processes and frameworks are in place, proportionate to the risk involved, to understand the extent and scale of the bias present. This enables mitigation of the group-skewed risks identified throughout the system. + +### Where it fails + +Bias is ubiquitous in AI systems, however negative impact concentrates where nobody is measuring, and a system built on a general-purpose model inherits skew its team never chose. + +- Inherited model bias propagates into every downstream decision the system takes. +- Application data, prompts, retrieval sources and thresholds amplify skew the base model did not have. +- Fine-tuning shifts behaviour in ways that go unnoticed without re-evaluation. + +### Practical controls + +Teams must combine quantitative tests with domain knowledge, and rely on processes rather than tools alone. + +- Run disaggregated behavioural evaluations across the groups the system affects. +- Review provider model documentation for known biases and evaluation gaps. +- Re-evaluate after fine-tuning, model updates and changes in deployment conditions. ## P03 — Explainability by Justification -```yaml -title: Explainability by Justification -description: Frontier models are black boxes; however AI systems that integrate them can be more opaque and undeterministic, or explainable through tooling, process and best practice. -commitment: We commit to develop tools and processes to enhance the transparency and explainability of AI systems where reasonable. -failure_modes: - - Unnecessarily convoluted agentic graphs - - Stated reasoning diverging from actual behaviour - - Lack of deterministic flows where systematic execution required -controls: - - Interpretability and attribution tooling - - Transparency artifacts such as model and system cards - - Infrastructure for semantic root cause analysis -related_links: - - XAI Framework - - "AI-RFX: transparency capabilities" -``` - -Frontier models are black boxes by design. When these are integrated into a larger AI system; these can become either more opaque or more transparent. This requires practitioners to work towards integrating and iteratively extending the infrastructure required to ensure key components have human interpretability and accountability. Explainability should also be proportionate to the risk involved in the system's outputs themselves. - -Explainability today is built from interpretability and attribution tooling, grounding outputs in citable sources, and transparency artifacts. Foundational capabilities involve model and system cards that state what a system is, what it was evaluated on and where it should not be used. Any trade-offs that involve explainability vs capability/performance, should be documented and weighed against the domain's need for justification. - -## P04 — Reproducibility & Provenance - -```yaml -title: Reproducibility & Provenance -description: Storing traces of a system does not ensure provenance. These should not only provide lineage but should be reproducible and auditable in order to be useful. -commitment: We commit to develop the infrastructure required to enable reproducibility and provenance across the operations of our AI systems. -failure_modes: - - Unpinned model versions - - Gaps in data governance throughout system - - Historical steps that cannot be reconstructed -controls: - - Version pinning for models, prompts and configuration - - Determinism in atomic steps through seed/input metadata - - Data provenance and lineage across operations -related_links: - - ML Maturity Model - - State of Production ML 2025 -``` - -Diagnosing a production incident requires reproducing the conditions that caused it. A team deploying a model it did not train cannot reproduce the model, but they can and must reproduce their own system. Foundational pillars of AI provenance include pinned model versions, the prompts and configuration in place, the data that flowed through, and the provenance of any fine-tune, including base model, data and adapters. Hosted models are updated and deprecated by their providers, so an unpinned version changes the system underneath its operators and must be explicitly captured. - -Reproducibility is also what makes auditability possible, not only lineage. For AI systems it is necessary to record not only decisions but also trajectory metadata required to reconstruct what specific steps previously carried out. Retaining these records is now an operational capability that is foundational for production best practices for any AI systems in production. +| Field | Content | +| :---- | :---- | +| Title | Explainability by Justification | +| Tagline | Frontier models are black boxes by design; however AI systems that integrate them can be explainable through tooling, process and best practice. | +| Commitment | We commit to develop tools and processes to enhance the transparency and explainability of AI systems where reasonable, enabling **Explainability by Justification**. | +| Failure modes | Unnecessarily convoluted agentic graphs
Stated reasoning diverging from actual behaviour
Lack of deterministic flows where systematic execution required | +| Practical controls | Interpretability and attribution tooling
Transparency artifacts such as model and system cards
Infrastructure for semantic root cause analysis | +| Related links | XAI Framework
AI-RFX: transparency capabilities | + +Frontier models are black boxes by design. When integrated into a larger AI system, they can become either more opaque or more transparent. This requires practitioners to work towards integrating and iteratively extending the infrastructure required to ensure key components have human interpretability and accountability. + +Similarly, explainability does not arise from tooling alone, but also from the processes and the steps carried out by domain experts. The right process should be designed proportionate to the risk involved in the system's outputs themselves. + +### Where it fails + +Opacity compounds when systems are assembled without explainability in mind, and the explanation a system gives is not automatically the truth of what it did. + +- Unnecessarily convoluted agentic graphs that no operator can follow end to end. +- Stated reasoning that diverges from the actual behaviour of the model. +- Missing deterministic flows where systematic, repeatable execution is required. + +### Practical controls + +Explainability today is built from interpretability and attribution tooling, grounding outputs in citable sources, and transparency artifacts. + +- Apply interpretability and attribution tooling proportionate to the risk of the output. +- Publish model and system cards that state what a system is, what it was evaluated on and where it should not be used. +- Build infrastructure for semantic root cause analysis, and document any explainability vs capability/performance trade-offs against the domain's need for justification. + +## P04 — Provenance by Reproducibility + +| Field | Content | +| :---- | :---- | +| Title | Provenance by Reproducibility | +| Tagline | Storing traces of a system does not ensure provenance. Lineage is not the goal but the means: operations that can be reproduced are what turn records into provenance, and provenance into auditability. | +| Commitment | We commit to develop the infrastructure that captures the lineage of our AI systems' operations, so historical steps can be reconstructed and reproduced, enabling **Provenance by Reproducibility**. | +| Failure modes | Unpinned model versions
Gaps in data governance throughout system
Historical steps that cannot be reconstructed | +| Practical controls | Version pinning for models, prompts and configuration
Determinism in atomic steps through seed/input metadata
Data provenance and lineage across operations | +| Related links | ML Maturity Model
State of Production ML 2025 | + +Diagnosing a production incident requires reproducing the conditions that caused it. A team deploying a model it did not train cannot reproduce the model, but they can and must be able to reproduce operations on their own system. Foundational pillars of AI provenance include pinned model versions, the prompts and configuration in place, the data that flowed through, and the provenance of any fine-tune, including base model, data and adapters. Hosted models are updated and deprecated by their providers, so an unpinned version changes the system underneath its operators and must be explicitly captured. + +Reproducibility is also what makes auditability possible, not only lineage. For AI systems it is necessary to record not only decisions but also trajectory metadata required to reconstruct what specific steps were previously carried out. Retaining these records is now an operational capability that is foundational for production best practices for any AI systems in production. + +### Where it fails + +A single missing trail can have a cascading impact on the provenance of the system, and the gap is usually discovered during the incident that needed it. + +- Unpinned model versions that let providers change the system underneath its operators. +- Gaps in data governance across the system's operations. +- Historical steps that cannot be reconstructed from what was recorded. + +### Practical controls + +Controls should make the team's own system reproducible even when the model it builds on is not theirs. + +- Pin versions for models, prompts and configuration, including the base model, data and adapters of any fine-tune. +- Make atomic steps deterministic through seed and input metadata. +- Record data provenance and lineage across operations, including the trajectory metadata needed to reconstruct past steps. ## P05 — Reskilling by Design -```yaml -title: Reskilling by Design -description: Automation at scale changes work across organisations and industries; those effects should be identified and planned for, not discovered. -commitment: We commit to identify, document and systematically support the re-skilling and evolution of our domain experts to mitigate the impact on workers affected by automation. -failure_modes: - - Lack of systematic plan for workforce support - - No retraining and reskilling paths - - Undocumented process change -controls: - - Workforce impact assessment - - Business change planning - - Process documentation -related_links: - - "AI-RFX: change management" -``` - -Automating medium or large processes affects many people across an organisation or industry, and general-purpose AI has widened the range of work this applies to. Technologists should look beyond the technology and help relevant stakeholders understand those effects and their impact. As Jevons observed of efficiency gains generally, cheaper automation tends to increase total demand for it not decrease it, so it is important that the industry as a whole is supported to evolve towards that increased demand. - -Even if more junior practitioners do not lead an operational transformation, they should help ensure that appropriate change-management processes are in place, particularly if they are involved in the development in the AI systems involved. Those processes should contribute towards a sustainable industry-wide effort to shift and evolve amid the accelerated transformation that AI systems are imposing in society and the workforce. - -## P06 — Monitoring & Evals - -```yaml -title: Monitoring & Evals -description: A system that is only evaluated before launch is unevaluated for most of its lifetime; robust observability and evaluation is required across the end-to-end lifecycle of the system. -commitment: We commit to evaluate our AI systems against the domain-specific consequences of their behaviour, and to ensure robust observability capapbilities are in place for as long as the system operates. -failure_modes: - - Benchmark contamination and overfitting - - Symmetric error assumptions - - Drift that goes undetected after deployment -controls: - - Domain-grounded capability and safety evals - - Cost-weighted metrics with production monitoring - - Incident feedback into the evaluation suite -related_links: - - "AI-RFX: practical benchmarks" - - State of Production ML 2025 -``` - -A system can score well on public benchmarks and still be wrong in ways that matter in its domain. Evals should be built from the consequences of each kind of error, including **capability evals** for what the system must do, **safety evals** for what it must not, and **cost-weighted measures** that reflect the asymmetry between a false alarm and a missed harm. - -In production the same questions continue once the system is in production and should be monitored with robust observability. Behaviour must be tracked against real traffic rather than curated test sets, and where relvevant advanced monitoring such as drift and outlier detection, as well as agent/model-as-judge patterns throughout the data flow of the systems.. Incidents and near misses should feed back into the evaluation suite, so each release is tested against the failures the previous one produced. +| Field | Content | +| :---- | :---- | +| Title | Reskilling by Design | +| Tagline | Automation at scale changes work across organisations and industries; those effects should be identified and planned for, not discovered. | +| Commitment | We commit to identify, document and systematically support the reskilling of our domain experts and mitigate the impact on workers affected by automation, enabling **Reskilling by Design**. | +| Failure modes | Lack of systematic plan for workforce support
No retraining and reskilling paths
Undocumented process change | +| Practical controls | Workforce impact assessment
Business change planning
Process documentation | +| Related links | AI-RFX: change management | + +Automating medium or large processes affects many people across an organisation or industry, and general-purpose AI has widened the range of work this applies to. Technologists should look beyond the technology and help relevant stakeholders understand those effects and their impact. As observed with Jevons' paradox, when a new technology unlocks major automations that reduce the costs of a service, total demand does not decrease; in fact, as the service becomes more accessible, total demand increases. So it is important that the industry as a whole is supported to evolve towards that increased demand. + +Even if more junior practitioners do not lead an operational transformation, they should contribute to ensure that appropriate change-management processes are in place, particularly if they are involved in the development of the AI systems involved. Those processes should contribute towards a sustainable industry-wide effort to shift and evolve amid the accelerated transformation that AI systems are imposing in society and the workforce. + +### Where it fails + +Displacement is rarely a single event; it accumulates while attention stays on the technology. + +- No systematic plan for supporting the workforce whose work is changing. +- No retraining or reskilling paths for the roles being automated. +- Process changes that go undocumented until the operational knowledge is gone. + +### Practical controls + +Controls should treat workforce impact as part of the delivery, not an afterthought of it. + +- Assess workforce impact before automating a process. +- Plan business change alongside technical delivery, with reskilling paths for affected domain experts. +- Document process changes so knowledge survives the transition. + +## P06 — Evaluation by Observability + +| Field | Content | +| :---- | :---- | +| Title | Evaluation by Observability | +| Tagline | A system that is only evaluated before launch is unevaluated for most of its lifetime; robust observability converts evals into a live property of the system. | +| Commitment | We commit to evaluate our AI systems against the domain-specific consequences of their behaviour, and to ensure robust monitoring capabilities are in place for as long as the system operates, enabling **Evaluation by Observability**. | +| Failure modes | Benchmark contamination and overfitting
Lack of telemetry, tracing, logging and general observability
Drift that goes undetected after deployment | +| Practical controls | Domain-grounded capability and safety evals
Cost-weighted metrics with production monitoring
Incident feedback into the evaluation suite | +| Related links | AI-RFX: practical benchmarks
State of Production ML 2025 | + +A system can score well on public benchmarks and still be wrong in ways that matter in its domain. Evals should be built from the consequences of each kind of error, including **capability evals** for what the system must do, **safety evals** for what it must not, and **cost-weighted measures** that reflect the asymmetry between a false alarm and a missed harm. + +The same questions continue once the system is in production and should be answered with robust observability. Behaviour must be tracked against real traffic rather than curated test sets, and where relevant advanced monitoring applied, such as drift and outlier detection, as well as agent/model-as-judge patterns throughout the data flow of the systems. + +### Where it fails + +Evaluation fails silently: the score stays green while the system and its conditions drift. + +- Benchmark contamination and overfitting inflating measured capability. +- No telemetry, tracing or logging in place to see behaviour after launch. +- Drift in data, usage or hosted model versions that goes undetected after deployment. + +### Practical controls + +Controls should make evaluation continuous, with production behaviour feeding the same suite that gated the release. + +- Build domain-grounded capability and safety evals with cost-weighted metrics. +- Monitor production behaviour with drift and outlier detection, and agent/model-as-judge patterns where relevant. +- Feed incidents and near misses back into the evaluation suite, so each release is tested against the failures the previous one produced. ## P07 — Trust by Privacy -```yaml -title: Trust by Privacy -description: Privacy protections must cover everyone a system touches, including people whose data reaches a model without their knowledge. -commitment: We commit to build and communicate processes that protect the data of stakeholders interacting with our AI systems directly and indirectly. -failure_modes: - - Training data memorisation and extraction - - Personal data retained by model providers - - Sensitive data accumulating in agent context -controls: - - Data minimisation before inference - - Provider retention terms and agreements - - Memorisation and extraction testing -related_links: - - "AI-RFX: privacy capabilities" -``` - -AI systems widen the paths along which personal data can leak. Models can memorise training data and reproduce it under extraction attacks, a builder-side risk that fine-tuners take on the moment they train on their own records. On the deployer side, every inference call can send user data to a third-party provider, so retention terms belong in the privacy assessment, and agents accumulate sensitive material in their working context as they operate on a person's behalf. - -Privacy should be designed into the system and its operating processes: minimise what enters a prompt or context window, contract for retention and use limits with providers, and test fine-tuned models for memorisation before release. Trust also requires communication, so stakeholders, direct and indirect, should be able to learn what data is held, how it is handled and why protecting it matters. - -## P08 — Security & Safety - -```yaml -title: Security & Safety -description: The attack surface of an AI system includes every piece of text its models read and every tool its agents can call. Security failures become safety failures the moment a system can act. -commitment: We commit to develop the processes and infrastructure required to secure our AI systems across their lifecycle, and to treat the safety of their actions as part of that security boundary. -failure_modes: - - Prompt injection turning content into instructions - - Agents misusing tools and permissions - - Model and data supply-chain compromise -controls: - - Threat reviews against the MLSecOps and OWASP agentic Top 10s - - Least-privilege, sandboxed tool access for agents - - Red-teaming, layered safeguards and monitoring -related_links: - - MLSecOps Top 10 - - OWASP Agentic Security (attack-vector report) -``` - -Security spans the full lifecycle of an AI system: training data can be poisoned, model artifacts and their dependencies compromised through the supply chain, and deployed models probed through their own interface. Prompt injection turns any text a model reads into a potential instruction channel, jailbreaks defeat behavioural safeguards, and an agent connected to tools converts a successful injection into real actions with real permissions. This is where security and safety meet: a compromised agent is not only a breached system but an unsafe one, and controls should match both the system's exposure and the harm its actions could cause. - -Teams should assume that capable adversaries will probe their models and tools, grant agents the least privilege their task requires, and validate layered safeguards through repeatable red-teaming and monitoring. The institute's MLSecOps Top 10 covers the pipeline-level vulnerabilities; the attack vectors specific to agentic systems are catalogued in the OWASP agentic security work the institute co-authored. Together with the OWASP Top 10 for LLM applications, these should anchor a threat review before any AI system reaches production. - -## P09 — Intent Alignment - -```yaml -title: Intent Alignment -description: An aligned system pursues the objectives its operators intended, within bounds set by the interests of the people its actions affect. Neither the intent nor the bounds can be assumed; both must be specified and tested. -commitment: We commit to evaluate whether our AI systems pursue the objectives their operators intended, to test for goal misgeneralisation, deception and misuse where reasonable, and to hold those objectives to the interests of the people they affect. -failure_modes: - - Goal misgeneralisation outside training conditions - - Deceptive behaviour under evaluation - - Operator objectives that harm the people affected -controls: - - Testable intent specification - - Adversarial and scheming evaluations - - Interruptible deployments with revisable objectives -related_links: - - K8s Agent OS (KAOS) - - Agentic safety programme -``` - -An aligned system behaves consistently with the purpose, constraints and values established by accountable people. A capable system can satisfy its evaluations while pursuing something other than what its operators intended: optimising a proxy that diverges outside training, behaving differently when it detects it is being tested, or being repurposed for harm. Alignment to operator intent is therefore the first requirement, and it must be made testable rather than assumed. - -It is not the last requirement. An objective can be faithfully pursued and still cause harm, so the values a system serves must extend beyond its operator to the people its actions affect. This principle asks what the system is pursuing and for whom; whether its errors fall unevenly across people is the separate question covered by principle 02. Teams should specify intent, examine how systems behave under adversarial pressure, and retain the ability to interrupt or revise deployments as real-world evidence changes. +| Field | Content | +| :---- | :---- | +| Title | Trust by Privacy | +| Tagline | A Frontier Model can hold data from people who have never used it; every layer built on top can multiply exposure, so model deployers must ensure privacy is designed at each one in order to enable trust. | +| Commitment | We commit to build and communicate privacy processes that protect the data of stakeholders interacting with our AI systems directly and indirectly, enabling **Trust by Privacy**. | +| Failure modes | Personal data extractable from a model by prompting
Sensitive data flowing to third parties the user never agreed to
One user's data surfacing in another user's session | +| Practical controls | Privacy impact assessment before connecting data to a model
Memory segregation across users with no shared session context
End-to-end encryption of personal data in transit and at rest | +| Related links | AI-RFX: privacy capabilities | + +AI systems widen the paths along which personal data can leak. Models can memorise training data and reproduce it under extraction attacks, a builder-side risk that fine-tuners take on the moment they train on their own records. On the deployer side, every inference call can send user data to third parties the person never agreed to, and shared agent infrastructure can let one user's data surface in another's session. + +Privacy should therefore be designed at every layer: assessed before data is connected to a model, segregated across users and tiers in memory, and encrypted in transit and at rest. Trust also requires communication, so stakeholders, direct and indirect, should be able to learn what data is held, how it is handled and why protecting it matters. + +### Where it fails + +The leak paths are wider than the application's own database, and most of them sit outside classic privacy reviews. + +- Personal data extractable from a model by prompting. +- Sensitive data flowing to third parties the person never agreed to. +- One user's data surfacing in another user's session. + +### Practical controls + +Controls should protect data at every layer it passes through, before it reaches a model, while it sits in memory, and wherever it travels. + +- Run a privacy impact assessment before connecting data to a model. +- Segregate memory across users, with no shared session context. +- Encrypt personal data end to end, in transit and at rest. + +## P08 — Safety by Security + +| Field | Content | +| :---- | :---- | +| Title | Safety by Security | +| Tagline | Any input a model processes is an attack surface; any tool an agent holds is blast radius. AI systems must be secured across their whole lifecycle, because there is no safety without security. | +| Commitment | We commit to develop the processes and infrastructure required to secure our AI systems across their lifecycle, and to treat the safety of their actions as part of that boundary, enabling **Safety by Security**. | +| Failure modes | Agents granted standing permissions beyond their task
Models, data and tools adopted without vetting or integrity checks
Agent credentials shared across users, tasks and environments | +| Practical controls | Threat modelling against the MLSecOps Top 10 and OWASP agentic attack vectors
Trust boundaries separating model instructions from external content
Red-teaming, layered safeguards and monitoring | +| Related links | MLSecOps Top 10
OWASP Agentic Security (attack-vector report) | + +Security spans the full lifecycle of an AI system: training data can be poisoned, model artifacts and their dependencies compromised through the supply chain, and deployed models probed through their own interface. Prompt injection turns any input a model processes into a potential instruction channel, jailbreaks defeat behavioural safeguards, and an agent connected to tools converts a successful injection into real actions with real permissions. This is where security and safety meet: a compromised agent is not only a breached system but an unsafe one, and controls should match both the system's exposure and the harm its actions could cause. + +Teams should assume that capable adversaries will probe their models and tools, and provision agents accordingly: least privilege, per-task credentials, and no standing access beyond need. The institute's MLSecOps Top 10 covers the pipeline-level vulnerabilities; the attack vectors specific to agentic systems are catalogued in the OWASP agentic security work the institute co-authored. Together with the OWASP Top 10 for LLM applications, these should anchor a threat review before any AI system reaches production. + +### Where it fails + +Most AI security failures are prepared in advance by the team's own defaults, long before an adversary arrives. + +- Agents granted standing permissions beyond what their task requires. +- Models, data and tools adopted without vetting or integrity checks. +- Agent credentials shared across users, tasks and environments. + +### Practical controls + +Controls should match the system's exposure and the harm its actions could cause. + +- Threat-model against the MLSecOps Top 10, the OWASP agentic attack vectors and the OWASP Top 10 for LLM applications. +- Enforce trust boundaries that separate model instructions from external content. +- Validate layered safeguards through repeatable red-teaming and monitoring. + +## P09 — Human Alignment by Intent + +| Field | Content | +| :---- | :---- | +| Title | Human Alignment by Intent | +| Tagline | A system optimises what it was given, not what was meant. Intent must be made explicit, tested under pressure, and revisable when evidence contradicts it. | +| Commitment | We commit to evaluate whether our AI systems pursue what was intended rather than what was measured, and to test for misgeneralisation, deception and misuse where reasonable, ensuring **Human Alignment by Intent**. | +| Failure modes | No definition of what the system must never pursue
High risk systems that cannot be interrupted once running
Objectives never revisited as the deployment context changes | +| Practical controls | Periodic objective reviews as deployment conditions change
Post-deployment audits of behaviour against the stated intent
Monitoring for divergence between the metric and intended outcome | +| Related links | K8s Agent OS (KAOS)
Agentic safety programme | + +An aligned system behaves consistently with the purpose, constraints and values established by accountable people. A capable system can satisfy its evaluations while pursuing something other than what its operators intended: optimising a proxy that diverges outside training, behaving differently when it detects it is being tested, or being repurposed for harm. Alignment to operator intent is therefore the first requirement, and that intent must be made explicit and testable rather than assumed. + +An objective can be faithfully pursued and still cause harm, so the values a system serves must extend beyond its operator to the people its actions affect, including a clear definition of what the system must never pursue. Those obligations do not end at deployment, as objectives should be revisited as deployment conditions change, behaviour audited against the stated intent, and high-risk systems kept interruptible for as long as they run. This principle asks what the system is pursuing and for whom; whether its errors fall unevenly across people is the separate question covered by principle 02. + +### Where it fails + +A system can pass its evaluations and still pursue the wrong thing, and the practices that allow it are set by the team long before deployment. + +- No definition of what the system must never pursue. +- High-risk systems that cannot be interrupted once running. +- Objectives never revisited as the deployment context changes. + +### Practical controls + +Controls should keep the stated intent and the deployed behaviour in contact for as long as the system runs. + +- Review objectives periodically as deployment conditions change. +- Audit post-deployment behaviour against the stated intent. +- Monitor for divergence between the metric and the intended outcome. diff --git a/.github/memory/2026-07-rebrand/research/content-proposals/12-sheet-open-source.md b/.github/memory/2026-07-rebrand/research/content-proposals/12-sheet-open-source.md index 2a20c53b..67892288 100644 --- a/.github/memory/2026-07-rebrand/research/content-proposals/12-sheet-open-source.md +++ b/.github/memory/2026-07-rebrand/research/content-proposals/12-sheet-open-source.md @@ -61,3 +61,209 @@ Fetched via GitHub REST API on the date of this sheet, then summarised by a smal - `EthicalML/XAI` — 1,255 stars, 185 forks, MIT. - `EthicalML/fml-security` (MLSecOps Top 10 code, referenced in `13-sheet-frameworks-reports.md`) — 125 stars, 23 forks, no license file found. - `axsaucedo/kaos` — API lookup returned 268 stars / 18 forks / Apache-2.0, but a full listing of the `EthicalML` org's repos (17 repos enumerated) found **no** repo with "kaos" or "agent" in the name — the KAOS project referenced across the prototype does not obviously correspond to a public repo under either account. **[OWNER: confirm KAOS's actual repo location before publishing any GitHub links or contributor/star counts for it.]** + +--- +# v3 — Comprehensive page specifications (research-grounded), for owner review + +Supersedes v2 (in git history). Every fact below is sourced from five research passes run 2026-07-31: source clones of kaos (main, 0.7.1-dev), KomputeProject/kompute, EthicalML/xai, both awesome lists (full reports in `tmp2/report-oss-kaos.md` and `tmp2/report-oss-others.md`), plus live docs crawls. Quotes are verbatim from the repos/docs. `[WIDGET: ...]` describes the bespoke component at that point; `[OWNER: ...]` marks a decision or confirmation. + +## Corrections v3 makes to claims currently on the site or in v2 + +| Current claim | Reality (sourced) | +| --- | --- | +| KAOS as "bounded agency" control plane with budgets/approval gates | KAOS is an agent orchestration system: five CRDs (Agent, ModelAPI, MCPServer, MemoryStore, AccessGrant), operator, CLI, UI, PAIS runtime. Its real governance surface is the v0.7.0 security plane (gateway authz, fail-closed OPA, AccessGrant) and scoped memory, not budget/approval primitives. | +| KAOS metrics 1.8k stars / 42 contributors / 160+ releases | 268 stars, v0.7.0 (31 Jul), Apache-2.0, K8s ≥1.28 (matches repos-metrics.yaml). | +| Production ML list "curated weekly", 7 categories with counts 48/37/... | 24 categories, ~551 entries (counted from README master); updates are monthly release summaries; the weekly thing is the newsletter (70k+ subscribers). | +| XAI "three steps" framing only | Confirmed ("the 3-steps of explainable machine learning"), but the README says built on "the 8 principles for Responsible Machine Learning" — legacy count. [OWNER: update the XAI README to nine principles, or keep site copy principle-count-neutral (v3 does the latter).] | +| Kompute "AMD, Intel, Nvidia, mobile GPUs" | README says "AMD, Qualcomm, NVIDIA & friends"; no Apple/MoltenVK/Intel claim exists in README or docs. v3 uses their vendor list. | +| KAOS quickstart YAML manifests from README | Source audit found the README/docs example manifests stale against the current API (removed fields, missing `spec.model`). The CLI command path is the one to show. [OWNER: worth fixing upstream in the kaos README too.] | + +## v3 /open-source (index) — "Project portal", not a card grid + +Addressing the owner comment: no reuse of the homepage showcase grid, and no four-projects paragraph. The index becomes a portal of full-width project panels with a persistent rail. + +[WIDGET: `ProjectPortal` — a scroll-driven vertical stack of four full-bleed panels, one per project, with a thin sticky rail on the left listing the four names (active one highlighted as the user scrolls, click jumps). Each panel is a consistent composition: status pill row (e.g. `v0.7.0 · APACHE-2.0 · K8S ≥1.28`), project wordmark-sized title, one-line subtitle, a live visual filling the panel's right/backdrop, and a single "Enter →" link to the detail page. The live visual is each project's own canvas or media, not a shared placeholder: KAOS = the kaos-graph canvas; Kompute = the KomputeCube canvas; XAI = a static imbalance-plot motif; Production ML list = an animated category constellation (see list page). Panels use the site's reveal/settle motion; the rail is the only new chrome. Reduced motion: panels render stacked and static. This is a new layout component, not `OpenSourceShowcase`; it expects exactly (pills, title, subtitle, visual, href) per project, which matches the owner's "header, subtitle, pill, image rendered in a nice way" framing.] + +> **Eyebrow:** OPEN SOURCE / BUILT IN PUBLIC +> **Title:** Open-source software and tools +> **Intro:** Openly licensed software for agent orchestration, GPU compute, explainability and production ML. Built in public, governed in the open, used in production. + +Panel subtitles (the only copy per panel): +> **KAOS** — Deploy, manage and orchestrate AI agents on Kubernetes. `v0.7.0 · APACHE-2.0 · K8S ≥1.28` +> **Kompute** — The general-purpose GPU compute framework for cross-vendor graphics cards, hosted by LF AI & Data. `LINUX FOUNDATION · APACHE-2.0 · 2.5K STARS` +> **XAI** — An explainability toolbox for machine learning, across data, model and production. `ALPHA · MIT · 1.3K STARS` +> **Awesome Production ML** — The community catalogue of production ML tooling: 24 categories, 550+ libraries. `COMMUNITY · 20.8K STARS` + +CTA at the portal's end: existing CenteredCTA ("Contribute to the open-source projects."). + +## v3 /open-source/kaos — flagship page + +> **Eyebrow:** OPEN SOURCE · V0.7.0 · APACHE-2.0 +> **Title:** K8s Agent OS (KAOS) +> **Intro:** Deploy, manage and orchestrate AI agents on Kubernetes. KAOS turns agents, models, tools and memory into declarative Kubernetes resources, reconciled by an operator, secured at the gateway and observable end to end. "Managing the chaos in your agentic systems." + +[WIDGET: keep the KAOS hero canvas (bare backdrop). The banner tagline above is from the project's own title card.] + +### From zero to a running agent + +> An agent is a resource, not a script. Install the system, deploy a model API and an agent, and talk to it; every agent exposes an OpenAI-compatible `/v1/chat/completions` endpoint, so existing clients work unchanged. + +[WIDGET: `QuickstartTerminal` — animated terminal in the site typewriter idiom, playing the real CLI sequence with realistic pauses and output lines, replay button, static code block under reduced motion: +`pip install kaos-cli` → `kaos system install` → `kaos modelapi deploy my-api --mode Hosted --model "smollm2:135m"` → `kaos agent deploy my-agent --modelapi my-api --model "smollm2:135m"` → `kaos agent invoke my-agent --message "Hello"` → `kaos ui`. +A secondary tab shows the Helm path (`helm repo add kaos https://axsaucedo.github.io/kaos/charts` → `helm install kaos kaos/kaos-operator -n kaos-system --create-namespace` → `kubectl apply -f my-agent.yaml`). Commands verified against the current CLI; the stale README YAML manifests are NOT reproduced.] + +### The control plane + +> Five custom resources define an agent system: **Agent** (the running agent, its model, tools, peers and memory binding), **ModelAPI** (LLM access, proxied via LiteLLM or hosted in-cluster with Ollama), **MCPServer** (tool servers speaking the Model Context Protocol), **MemoryStore** (the central memory service agents bind to) and **AccessGrant** (who may reach what). Four controllers reconcile them into pods, services and policy. + +[WIDGET: `ControlPlaneMap` — a bespoke interactive canvas: the five CRDs as typed nodes around an operator hub, with the reconciled runtime (agent pods, gateway, memory service) as a second ring. Selecting a node slides a side panel with the CRD's one-line definition and two or three key spec fields (from the source audit). A delegation pulse traces Agent → MCPServer and Agent → Agent (A2A) paths on demand, user-triggered, never auto-rotating. This is the evolution of kaos-graph into a real architecture explorer; reduced motion gets the static graph with the panel interaction intact.] + +> Agents delegate through the A2A protocol: discovery via `/.well-known/agent.json`, sub-agents exposed to the model as tools, and "hierarchical agent systems with automatic delegation". + +### Memory that survives the session + +> "Memory is augmentation, not a hard dependency." A MemoryStore runs one central memory service per store, with Mem0 embedded as a library rather than a separate server. Short-term memory keeps the recent conversation window in relational rows; an optional rolling summary preserves narrative continuity; long-term memory holds semantic facts in a vector store, recalled by relevance across sessions. + +> Scopes make memory multi-tenant: reads select session, agent, user or store scope, and "scope is derived server-side from the authenticated agent identity", never from model arguments. When a store fails, the agent keeps serving short-term-only and surfaces a `MemoryDegraded` condition instead of going down. + +[WIDGET: `MemoryTiers` — a three-band diagram (short-term rows / rolling digest / long-term vector recall) with an animated event flowing through: a user message lands in the short-term band, the digest band absorbs it into the summary, and a fact crystallises into the long-term band. A scope selector (session / agent / user / store) dims the bands a read at that scope may touch. Static bands with the scope selector under reduced motion. Data and labels straight from the docs' own tier table.] + +Storage fact row (static): `LOCAL: Chroma + SQLite on one volume · EXTERNAL: pgvector + Postgres, 2 replicas` + +### Secured at the gateway + +> Security is off until you turn it on, and fail-closed once you do: protected routes pass Envoy Gateway, which verifies JWTs and consults an OPA policy decision point; "an unavailable PDP never permits a request". Identity is two-plane: the subject (a Keycloak user, or the agent itself when autonomous) and the acting agent, verified on every hop. Agent identity comes from per-agent ServiceAccounts with short-lived projected tokens, an identity broker, or per-agent OAuth clients registered dynamically. + +> Authorization is declarative: `AccessGrant` resources bind users, groups and agents to the resources they may enter, and the operator projects them into policy. Memory enforces the same identities, so one user's recall can never serve another's session. + +[WIDGET: `RequestPath` — a horizontal flow: client → Envoy Gateway (JWT check) → OPA PDP (allow/deny stamp) → Agent → onward hop with both tokens shown as small labelled chips (subject / actor). A toggle switches the scenario: user request, autonomous agent, denied request (red 403 path). Step-through on click, not timed. Static diagram under reduced motion.] + +### Watch it work + +[WIDGET: `MediaBand` — the two real docs GIFs, self-hosted copies: `demo.gif` (the dashboard workflow) and `demo-monitoring.gif` (the observability view with traces and flamegraphs). Lazy-loaded, poster first frames, captions linking to the docs. The UI screenshot set (agent chat, memory inspector, MCP tool debugger) exists if we prefer stills to GIFs on mobile. [OWNER: confirm self-hosting copies of these assets.]] + +> The dashboard covers the operational loop: monitor agents, chat with them live, inspect memory sessions and events, debug MCP tools with real calls, and stream pod logs. Agents ship OpenTelemetry instrumentation, so traces land in your existing observability stack. + +### Metrics + CTAs + +Metrics row (repos-metrics.yaml): `268 STARS · V0.7.0 · APACHE-2.0 · K8S ≥1.28` +CTAs: "Explore the project →" github.com/axsaucedo/kaos · "Read the docs →" axsaucedo.github.io/kaos [OWNER: confirm repo stays under axsaucedo at cutover.] + +## v3 /open-source/kompute + +> **Eyebrow:** OPEN SOURCE · LINUX FOUNDATION · APACHE-2.0 +> **Title:** Kompute +> **Intro:** "The general purpose GPU compute framework for cross vendor graphics cards (AMD, Qualcomm, NVIDIA & friends). Blazing fast, mobile-enabled, asynchronous, and optimized for advanced GPU acceleration usecases." Donated by the Institute and "backed by the Linux Foundation as a hosted project by the LF AI & Data Foundation." + +[WIDGET: KomputeCube canvas as the hero backdrop (existing shared canvas, bare full-bleed variant).] + +### The mental model + +> Four concepts carry the whole framework: a **Manager** owns the device, **Tensors** own data, an **Algorithm** wraps the shader logic, and a **Sequence** records operations to submit as a batch. Everything else is Vulkan doing what Vulkan does, without the boilerplate. + +[WIDGET: `SequencePipeline` — an animated strip showing a sequence being recorded then dispatched: tensor chips load, an OpSyncDevice pulse moves them to the device band, OpAlgoDispatch fires the shader block, results sync back. Mirrors the real API call order from the README example. One pass on scroll-into-view, replayable; static diagram (the project's own kompute-architecture.jpg as fallback imagery) under reduced motion.] + +### Two languages, one API + +[WIDGET: `CodeTabs` — tabbed Python / C++ block with the verbatim README examples (the mgr/tensor/algorithm/sequence flow with push constants), syntax highlighted, copy button. Static.] + +> The Python module covers experimentation; the C++ SDK goes as deep as the optimisation requires. Asynchronous and parallel execution comes through GPU family queues, and the BYOV design ("bring-your-own-Vulkan") plugs into existing Vulkan applications. + +### Proven where it counts + +> Kompute backs on-device LLM inference in **GPT4ALL**, appeared as a backend in **llama.cpp**, and powers **vkJAX**, the JAX interpreter for Vulkan. It runs on Android via the NDK, on Raspberry Pi through Mesa, and in the Godot engine for game development. [OWNER: llama.cpp's README labels its Kompute backend "now decomissioned"; phrasing here says "appeared", confirm you're happy naming it.] + +Fact strip: `90% UNIT TEST COVERAGE · MONTHLY COMMUNITY CALLS · FOSDEM TALKS · C++ AND PYTHON` + +Metrics row: `2.5K GITHUB STARS · 42 CONTRIBUTORS · 197 FORKS` +CTAs: "View on GitHub →" github.com/KomputeProject/kompute · "kompute.cc →" + +## v3 /open-source/xai + +> **Eyebrow:** OPEN SOURCE · ALPHA · MIT +> **Title:** XAI — an eXplainability toolbox for machine learning +> **Intro:** "XAI is a Machine Learning library that is designed with AI explainability in its core." It implements the three steps of explainable machine learning: data analysis, model evaluation and production monitoring, tools and process together. + +### The toolbox, by stage + +[WIDGET: `StageExplorer` — three-stage selector in the principles-explorer interaction family but purpose-built for an API: selecting a stage shows its real functions as rows, each row expanding to a one-line purpose and a thumbnail of the actual README plot. Data below is the verified public API:] + +> **01 — Data analysis.** `imbalance_plot` (class imbalances across one or more columns) · `balance` (up/down-sampling to correct them) · `correlations` (matrix or dendogram) · `group_by_columns` · `balanced_train_test_split` (splits balanced across protected attributes). +> **02 — Model evaluation.** `feature_importance` (permutation-based) · `metrics_plot` (metric imbalances overall and across protected columns) · `roc_plot` (ROC per group, e.g. gender, ethnicity, age) · `confusion_matrix_plot`. +> **03 — Production monitoring.** `smile_imbalance` (accuracy and statistical metrics bucketed by prediction probability) with explicit support to "visualise benefits of adding manual review on probability thresholds". + +> The library ships with a census dataset for experimentation (`xai.data.load_census()`), and installs from PyPI. The protected-attribute lens runs through every stage: the same gender or ethnicity split that shapes the train/test split reappears in the metric and ROC comparisons. + +[WIDGET: `ImbalanceDemo` (optional): a live rendering of the library's signature plot, class-imbalance bars across a protected column with an "apply balancing" toggle animating to the corrected distribution, data precomputed from the bundled census dataset so the chart is real, not invented. Cut if it reads as decoration.] + +### Talk and worked example + +> The approach was presented at PyData London, covering how to identify and mitigate undesired bias with the toolbox. [WIDGET: video card with the talk thumbnail from the README, linking out. [OWNER: confirm the PyData 2019 talk is the video you want surfaced; the legacy site's credit-risk video reference appears to be this talk.]] + +Metrics row: `1.3K STARS · MIT · ALPHA — ACTIVE` +CTA: "Open the framework →" github.com/EthicalML/xai · "Read the docs →" ethicalml.github.io/xai + +## v3 /open-source/production-ml-list + +> **Eyebrow:** OPEN SOURCE · COMMUNITY · 20.8K STARS +> **Title:** Awesome Production Machine Learning +> **Intro:** "A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning." More than 550 libraries across 24 categories, with a summary of new additions released every month. + +### Explore the categories + +[WIDGET: `CategoryAtlas` — the page's centrepiece: all 24 real categories as a packed, animated field sized by entry count (Evaluation & Monitoring 68 · Deployment & Serving 60 · Computation Optimisation 58 · Industry NLP 45 · Reinforcement Learning 39 · Training & Orchestration 27 · and the rest), each tile carrying the README's own emoji and count. Hover/tap lifts a tile and shows its description; clicking goes to the section anchor on GitHub. Counts are generated at build time by parsing the README (small script, refreshed each deploy), so the numbers stay honest without manual upkeep. Reduced motion: a static two-column table of category, count. [OWNER: confirm build-time README parsing; the alternative is names without counts.]] + +### How it stays current + +> New libraries are added continuously and summarised in monthly releases, so watching the repository doubles as an ecosystem changelog. The weekly companion is the Machine Learning Engineer newsletter, where "over 70,000 ML professionals and enthusiasts" receive curated production ML articles and tutorials. + +[WIDGET: `StarHistory` — a single static SVG line of the star history (the README embeds the star-history chart; we render our own from a build-time snapshot rather than hotlinking). Optional; cut if the page feels busy.] + +Metrics row: `20.8K STARS · 550+ LIBRARIES · 24 CATEGORIES` (all build-time derived or repos-metrics.yaml) +CTA: "Browse the ecosystem on GitHub →" github.com/EthicalML/awesome-production-machine-learning + +## v3 /open-source/ai-guidelines + +> **Eyebrow:** OPEN SOURCE · POLICY +> **Title:** Awesome AI Regulation, Principles & Guidelines +> **Intro:** A repository "mapping the ecosystem of guidelines, principles, codes of ethics, standards and regulation being put in place around artificial intelligence", from national law to practical checklists. + +### Two ways in + +> By geography: national regulation and policy across fifteen economic areas, from the EU AI Act ("the first regulation implemented on AI") to national strategies across the Americas, Asia and the Middle East. By theme: high-level frameworks and principles, industry standards initiatives, practical tools and checklists, courses, and research newsletters. + +[WIDGET: `RegulationGrid` — a compact grid of the fifteen economic areas (flag + name), each linking to its section anchor on GitHub, with the six thematic sections as a second row of pills. Static component with hover states only; this page stays light deliberately, the catalogue is the content. Generated from the README structure at build time alongside the CategoryAtlas parser.] + +> The Institute's own nine principles sit in the same ecosystem this list maps, which is why the repository is where our policy work and the wider landscape meet. + +Metrics row: `1.4K STARS · 15 ECONOMIC AREAS · MIT` +CTA: "Browse AI guidelines →" github.com/EthicalML/awesome-artificial-intelligence-regulation + +## Build inventory (bespoke components, no reuse-for-reuse's-sake) + +| Component | Page | What it shows | Interaction | Reduced motion | +| --- | --- | --- | --- | --- | +| ProjectPortal | index | four full-bleed project panels + sticky rail | scroll-driven rail highlight, jump links | stacked static panels | +| QuickstartTerminal | kaos | real CLI quickstart, Helm tab | autoplay once, replay, tab switch | static code block | +| ControlPlaneMap | kaos | five CRDs + operator + runtime ring | node select → spec panel, on-demand delegation pulse | static graph, panels intact | +| MemoryTiers | kaos | three memory bands + scope selector | event flow animation, scope dimming | static bands + selector | +| RequestPath | kaos | gateway → PDP → agent token flow | scenario toggle, click step-through | static diagram | +| MediaBand | kaos | demo.gif, demo-monitoring.gif | lazy, poster frames | posters only | +| SequencePipeline | kompute | record → dispatch → sync flow | one pass on view, replay | architecture still | +| CodeTabs | kompute | verbatim Python/C++ examples | tabs, copy | n/a | +| StageExplorer | xai | 3 stages → real API rows → plot thumbnails | stage select, row expand | all stages stacked | +| ImbalanceDemo (optional) | xai | real census imbalance plot + balance toggle | toggle | final state | +| CategoryAtlas | production-ml-list | 24 categories sized by real counts | hover lift, click-out | two-column table | +| StarHistory (optional) | production-ml-list | star growth line | none | static | +| RegulationGrid | ai-guidelines | 15 areas + 6 themes | hover, click-out | static | + +Shared modules only where two consumers exist: the terminal reuses the typewriter core; ControlPlaneMap and the index's kaos panel share the kaos-graph canvas engine; the README-parser build script feeds both CategoryAtlas and RegulationGrid. Everything else is a colocated custom element per the client-architecture order. No islands: nothing here changes rendered structure from state. + +## Open owner decisions (collected) + +1. Index portal: approve the ProjectPortal direction (full-bleed panels + rail) before any build. +2. KAOS: self-hosting the docs GIFs/screenshots; repo location at cutover; whether to fix the stale README manifests upstream. +3. XAI: update the repo README's "8 principles" to nine, or keep site copy count-neutral; confirm the PyData talk as the surfaced video. +4. Kompute: naming llama.cpp given the "decomissioned" backend note. +5. Production ML list: approve build-time README parsing for live counts (recommended) vs static names. +6. Sequencing: KAOS page first (release momentum), then index portal, then the three remaining pages. diff --git a/CONVENTIONS.md b/CONVENTIONS.md index 84d9e75c..68089d2c 100644 --- a/CONVENTIONS.md +++ b/CONVENTIONS.md @@ -8,7 +8,9 @@ The ratified decisions in `.github/memory/2026-07-rebrand/` are authoritative. A - Use a content collection when the site validates and iterates a set. Principles, partners, survey questions, and repository metrics are collections defined in `src/content.config.ts`. - Page-owned prose, headings, links, lists, and configuration belong in that page's MDX front matter or body. - Components arrange supplied content. They do not hide owner-editable page copy. +- Frontmatter copy fields render as escaped plain text by default. When a field genuinely needs inline emphasis, the standard is inline HTML in the field rendered through `set:html` at the consuming component; do not add markdown filters for frontmatter, and do not enable `set:html` on a field before its content actually carries markup. Scope stays inline (``, ``) on first-party copy fields only, never links or block markup. Not currently enabled anywhere. - `composed: true` means the MDX owns its full-width composition. Otherwise `ProseLayout.astro` supplies the standard article shell. +- External HTTP(S) links outside `ethical.institute` and its subdomains open in a new tab with `target="_blank" rel="noopener noreferrer"`; internal, relative, anchor, and `mailto:` links do not. Markdown links are enforced by `src/plugins/rehype-external-links.mjs`; component-authored anchors must apply the same attributes. The placement rule is: content lives with its owner; a separate file must be earned by two unrelated consumers, a validated set, or a genuinely large dataset. @@ -68,6 +70,11 @@ All first-party client behaviour is TypeScript. Do not add first-party runtime J | Survey title morph | Homepage report heading → survey explorer hero | `MorphPairs`, `ReportsSection`, `ArticleHero` | | Initiative title morphs | Homepage maturity / security card titles → matching framework heroes | `MorphPairs`, `ReportsSection`, `ArticleHero`, `BaseLayout` | | KAOS title morph | Homepage KAOS card title → KAOS page hero h1 | `MorphPairs`, `OpenSourceShowcase`, `ArticleHero` | +| OSS portal title morphs | `/open-source/` panel titles → project page heroes | `MorphPairs`, `ProjectPortal`, `ArticleHero` | +| Project portal activity | `/open-source/` project panels and production ML constellation | `ProjectPortal`, `CategoryConstellation`, `CanvasEngine` | +| Kompute sequence playback | Kompute detail sequence pipeline | `SequencePipeline` | +| Shared animation playback | Animated windows entering/leaving the viewport | `AnimationWindow`, slotted animation owner | +| Production ML atlas drift | Production ML catalogue category field | `CategoryAtlas` | | Partner logo morph | Clicked marquee instance → matching directory logo | `MorphPairs`, `AffiliationMarquee`, `PartnerDirectory` | | Principle directional slide | `/principles/NN/` prev/next → adjacent principle | `PrincipleLayout`, `Motion`, `tokens.css` | diff --git a/astro.config.mjs b/astro.config.mjs index 842066b9..c1d43eb7 100644 --- a/astro.config.mjs +++ b/astro.config.mjs @@ -3,6 +3,7 @@ import mdx from '@astrojs/mdx'; import sitemap from '@astrojs/sitemap'; import { unified } from '@astrojs/markdown-remark'; import preact from '@astrojs/preact'; +import rehypeExternalLinks from './src/plugins/rehype-external-links.mjs'; import rehypeSectionize from './src/plugins/rehype-sectionize.mjs'; export default defineConfig({ @@ -36,7 +37,7 @@ export default defineConfig({ '/privacypolicy.html': '/privacy/', }, markdown: { - processor: unified({ rehypePlugins: [rehypeSectionize] }), + processor: unified({ rehypePlugins: [rehypeExternalLinks, rehypeSectionize] }), }, integrations: [mdx(), preact(), sitemap()], }); diff --git a/public/images/oss/demo-monitoring-poster.webp b/public/images/oss/demo-monitoring-poster.webp new file mode 100644 index 00000000..5496ef2d Binary files /dev/null and b/public/images/oss/demo-monitoring-poster.webp differ diff --git a/public/images/oss/demo-monitoring.gif b/public/images/oss/demo-monitoring.gif new file mode 100644 index 00000000..57cc2c53 Binary files /dev/null and b/public/images/oss/demo-monitoring.gif differ diff --git a/public/images/oss/demo-poster.webp b/public/images/oss/demo-poster.webp new file mode 100644 index 00000000..f584e7c8 Binary files /dev/null and b/public/images/oss/demo-poster.webp differ diff --git a/public/images/oss/demo.gif b/public/images/oss/demo.gif new file mode 100644 index 00000000..77bb08ae Binary files /dev/null and b/public/images/oss/demo.gif differ diff --git a/public/images/oss/kompute-architecture.jpg b/public/images/oss/kompute-architecture.jpg new file mode 100644 index 00000000..b060de3d Binary files /dev/null and b/public/images/oss/kompute-architecture.jpg differ diff --git a/public/images/oss/xai-imbalance.jpg b/public/images/oss/xai-imbalance.jpg new file mode 100644 index 00000000..93b75d18 Binary files /dev/null and b/public/images/oss/xai-imbalance.jpg differ diff --git a/public/images/oss/xai-metrics.jpg b/public/images/oss/xai-metrics.jpg new file mode 100644 index 00000000..ed31e4cc Binary files /dev/null and b/public/images/oss/xai-metrics.jpg differ diff --git a/public/images/oss/xai-monitoring.png b/public/images/oss/xai-monitoring.png new file mode 100644 index 00000000..6c3b9b0e Binary files /dev/null and b/public/images/oss/xai-monitoring.png differ diff --git a/public/images/oss/xai-pydata-talk.jpg b/public/images/oss/xai-pydata-talk.jpg new file mode 100644 index 00000000..1ffa8eae Binary files /dev/null and b/public/images/oss/xai-pydata-talk.jpg differ diff --git a/scripts/parse-awesome-lists.mjs b/scripts/parse-awesome-lists.mjs new file mode 100644 index 00000000..1c1e525e --- /dev/null +++ b/scripts/parse-awesome-lists.mjs @@ -0,0 +1,160 @@ +import { spawnSync } from 'node:child_process'; +import { writeFileSync } from 'node:fs'; +import { fileURLToPath } from 'node:url'; +import path from 'node:path'; + +const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..'); +const outputPath = path.join(root, 'src/content/oss-catalogues.json'); + +const repositories = { + productionMl: 'EthicalML/awesome-production-machine-learning', + aiGuidelines: 'EthicalML/awesome-artificial-intelligence-regulation', +}; + +// Refresh the committed offline snapshot with: node scripts/parse-awesome-lists.mjs +function fetchReadme(repository) { + for (const branch of ['master', 'main']) { + const url = `https://raw.githubusercontent.com/${repository}/${branch}/README.md`; + const result = spawnSync('curl', ['-fsSL', '--retry', '1', url], { + encoding: 'utf8', + maxBuffer: 10 * 1024 * 1024, + }); + if (result.status === 0 && result.stdout.trim()) return { branch, text: result.stdout, url }; + } + throw new Error(`Could not fetch README for ${repository} from master or main.`); +} + +function sliceBetween(text, start, end) { + const startIndex = text.indexOf(start); + const endIndex = text.indexOf(end, startIndex + start.length); + if (startIndex < 0 || endIndex < 0) throw new Error(`Missing README boundary: ${start} / ${end}`); + return text.slice(startIndex, endIndex); +} + +function markdownLinks(text) { + return [...text.matchAll(/\[([^\]]+)\]\(#([^)]+)\)/g)].map(([, label, anchor]) => ({ + label: label.trim(), + anchor: `#${anchor}`, + })); +} + +function sections(text, headingLevel) { + const lines = text.split(/\r?\n/); + const heading = new RegExp(`^#{${headingLevel}}\\s+(.+?)\\s*$`); + const results = []; + let current; + + for (const line of lines) { + const match = line.match(heading); + if (match) { + current = { heading: match[1], count: 0 }; + results.push(current); + } else if (current && /^\s*[*+-]\s+\[/.test(line)) { + current.count += 1; + } + } + return results; +} + +function splitLeadingEmoji(label) { + const match = label.match(/^(\S+)\s+(.+)$/u); + if (!match) throw new Error(`Expected an emoji before label: ${label}`); + return { emoji: match[1], name: match[2] }; +} + +function splitTrailingEmoji(label) { + const match = label.match(/^(.+?)\s+(\S+)$/u); + if (!match) throw new Error(`Expected an emoji after label: ${label}`); + return { name: match[1], emoji: match[2] }; +} + +function zipLinksAndCounts(links, counts, parseLabel) { + if (links.length !== counts.length) { + throw new Error( + `README structure mismatch: found ${links.length} links and ${counts.length} sections.`, + ); + } + const used = new Set(); + const normalise = (value) => + value + .toLowerCase() + .replaceAll('&', 'and') + .replaceAll(/[^a-z0-9]+/g, ' ') + .trim(); + + return links.map(({ label, anchor }, index) => { + const parsed = parseLabel(label); + let countIndex = counts.findIndex( + ({ heading }, candidate) => + !used.has(candidate) && normalise(heading) === normalise(parsed.name), + ); + if (countIndex < 0 && !used.has(index)) countIndex = index; + if (countIndex < 0) throw new Error(`Could not match README section for ${parsed.name}.`); + used.add(countIndex); + return { ...parsed, count: counts[countIndex].count, anchor }; + }); +} + +const production = fetchReadme(repositories.productionMl); +const productionLinks = markdownLinks( + sliceBetween( + production.text, + '## Quick links to sections on this page', + '## Contributing to the list', + ), +); +const productionSections = sections( + sliceBetween(production.text, '# Main Content', '# Other Awesome Lists'), + 2, +); +const categories = zipLinksAndCounts(productionLinks, productionSections, splitLeadingEmoji); + +const regulation = fetchReadme(repositories.aiGuidelines); +const areaLinks = markdownLinks( + sliceBetween(regulation.text, '### National Regulation by Economic Area', '### Other Sections'), +); +const areaSections = sections( + sliceBetween( + regulation.text, + '# Regulation and Policy', + '# High Level Frameworks and Principles', + ), + 2, +); +const areas = zipLinksAndCounts(areaLinks, areaSections, splitTrailingEmoji); + +const themeLinks = markdownLinks( + sliceBetween(regulation.text, '### Other Sections', '## Other relevant resources'), +); +const themeSections = sections( + regulation.text.slice(regulation.text.indexOf('# High Level Frameworks and Principles')), + 1, +); +const themes = zipLinksAndCounts(themeLinks, themeSections, splitLeadingEmoji); + +if (categories.length !== 24 || areas.length !== 15 || themes.length !== 6) { + throw new Error( + `Unexpected catalogue sizes: ${categories.length} categories, ${areas.length} areas, ${themes.length} themes.`, + ); +} + +const snapshot = { + productionMl: { + repository: `https://github.com/${repositories.productionMl}`, + source: production.url, + branch: production.branch, + categories, + }, + aiGuidelines: { + repository: `https://github.com/${repositories.aiGuidelines}`, + source: regulation.url, + branch: regulation.branch, + areas, + themes, + }, +}; + +writeFileSync(outputPath, `${JSON.stringify(snapshot, null, 2)}\n`); +console.log( + `Wrote ${path.relative(root, outputPath)} (${categories.length} categories, ${areas.length} areas, ${themes.length} themes).`, +); diff --git a/src/components/AnimationWindow.astro b/src/components/AnimationWindow.astro new file mode 100644 index 00000000..4db0bcee --- /dev/null +++ b/src/components/AnimationWindow.astro @@ -0,0 +1,194 @@ +--- +interface Props { + title?: string; + chromeOnly?: boolean; + class?: string; +} + +const { title, chromeOnly = false, class: className }: Props = Astro.props; +--- + + +
+ + {title} + + + { + !chromeOnly && ( + + ) + } + +
+
+
+ + + + + diff --git a/src/components/ArticleHero.astro b/src/components/ArticleHero.astro index 194186f1..52ab56e6 100644 --- a/src/components/ArticleHero.astro +++ b/src/components/ArticleHero.astro @@ -1,12 +1,14 @@ --- import ArticleHeroCanvas from './ArticleHeroCanvas.astro'; import KaosGraph from './KaosGraph.astro'; +import KomputeCube from './KomputeCube.astro'; const { eyebrow, title, intro, canvas = false, transitionName } = Astro.props; --- -
- {canvas === 'kaos' ? : canvas && } +
+ {canvas === 'kaos' ? : canvas === true && } + {canvas === 'kompute' && }

{eyebrow}

{title}

{intro &&

{intro}

} diff --git a/src/components/CategoryAtlas.astro b/src/components/CategoryAtlas.astro new file mode 100644 index 00000000..9c87bbb9 --- /dev/null +++ b/src/components/CategoryAtlas.astro @@ -0,0 +1,255 @@ +--- +interface Category { + name: string; + emoji: string; + count: number; + anchor: string; +} + +interface Props { + categories: Category[]; + repository: string; +} + +const { categories, repository } = Astro.props; +const maximum = Math.max(...categories.map(({ count }) => count)); +--- + + +
+ { + categories.map(({ name, emoji, count, anchor }, index) => { + const size = 104 + Math.round((count / maximum) * 94); + const span = count >= 39 ? 2 : 1; + return ( + + + {name} + {count} + Browse {count} entries in this section. + + ); + }) + } +
+ +
+
CategoryCount
+ { + categories.map(({ name, count }) => ( +
+ <> + {name} + {count} + +
+ )) + } +
+
+ + + + diff --git a/src/components/CodeTabs.astro b/src/components/CodeTabs.astro new file mode 100644 index 00000000..7887861e --- /dev/null +++ b/src/components/CodeTabs.astro @@ -0,0 +1,140 @@ +--- +import { Code } from 'astro:components'; +import AnimationWindow from './AnimationWindow.astro'; + +interface Props { + python: string; + cpp: string; +} + +const { python, cpp } = Astro.props; +--- + + + + +
+ + +
+
+ +
+
+ + + + diff --git a/src/components/ControlPlaneMap.astro b/src/components/ControlPlaneMap.astro new file mode 100644 index 00000000..bcfe5405 --- /dev/null +++ b/src/components/ControlPlaneMap.astro @@ -0,0 +1,201 @@ +--- +const resources = [ + { + name: 'Agent', + definition: 'The running agent, its model, tools, peers and memory binding.', + fields: ['spec.model', 'spec.modelApiRef', 'spec.mcpServers / spec.subAgents'], + }, + { + name: 'ModelAPI', + definition: 'LLM access, proxied through LiteLLM or hosted in-cluster with Ollama.', + fields: ['spec.mode', 'spec.model', 'spec.provider'], + }, + { name: 'MCPServer', definition: 'A tool server speaking the Model Context Protocol.', fields: ['spec.transport', 'spec.url', 'spec.command'] }, + { name: 'MemoryStore', definition: 'The central memory service that agents bind to.', fields: ['spec.backend', 'spec.vectorStore', 'spec.storage'] }, + { + name: 'AccessGrant', + definition: 'The declarative binding between identities and reachable resources.', + fields: ['spec.subjects', 'spec.resources', 'spec.verbs'], + }, +]; +--- + + +
+ + +
+
+
+ { + resources.map(({ name }, index) => ( + + )) + } +
+
+ { + resources.map(({ name, definition, fields }, index) => ( +
+

KIND · {name}

+

{name}

+

{definition}

+
+ {fields.map((field) => ( + <> +
{field}
+
key spec field
+ + ))} +
+
+ )) + } +
+ +
+
+ + + + diff --git a/src/components/KaosGraph.astro b/src/components/KaosGraph.astro index 7d5c131d..bc501c13 100644 --- a/src/components/KaosGraph.astro +++ b/src/components/KaosGraph.astro @@ -21,7 +21,12 @@ const transitionStyle = transitionName ? `view-transition-name: ${transitionName data-morph-source={morphSource} /> ) : embedded ? ( - + ) : (

CUSTOM WIDGET

diff --git a/src/components/KaosProjectFooter.astro b/src/components/KaosProjectFooter.astro new file mode 100644 index 00000000..2ab0cd89 --- /dev/null +++ b/src/components/KaosProjectFooter.astro @@ -0,0 +1,94 @@ +--- +import { getEntry } from 'astro:content'; + +interface Metric { + value: string; + label: string; +} +interface Link { + href: string; + label: string; +} +interface Props { + links: Link[]; +} +const { links }: Props = Astro.props; +const metricsEntry = await getEntry('metrics', 'kaos'); +const metrics: Metric[] = (metricsEntry?.data.values ?? []).filter(({ label }) => label !== 'CONTRIBUTORS'); +--- + + + + diff --git a/src/components/KomputeCube.astro b/src/components/KomputeCube.astro new file mode 100644 index 00000000..6d8e6afe --- /dev/null +++ b/src/components/KomputeCube.astro @@ -0,0 +1,39 @@ +--- +interface Props { + embedded?: boolean; + backdrop?: boolean; +} + +const { embedded = false, backdrop = false } = Astro.props; +--- + +{ + embedded ? ( + + + + ) : ( +
+

CUSTOM WIDGET

+

Kompute cube

+ + + +
+ ) +} + + + + diff --git a/src/components/MediaBand.astro b/src/components/MediaBand.astro new file mode 100644 index 00000000..8cc455de --- /dev/null +++ b/src/components/MediaBand.astro @@ -0,0 +1,99 @@ +--- +interface MediaItem { + alt: string; + caption: string; + href: string; + poster: string; + src: string; +} +interface Props { + items: MediaItem[]; +} +const { items }: Props = Astro.props; +--- + + + { + items.map(({ alt, caption, href, poster, src }) => ( +
+ {alt} +
+ + {caption} → + +
+
+ )) + } +
+ + + + diff --git a/src/components/MemoryTiers.astro b/src/components/MemoryTiers.astro new file mode 100644 index 00000000..b7c65d55 --- /dev/null +++ b/src/components/MemoryTiers.astro @@ -0,0 +1,214 @@ +--- +interface Props { + storage: string; +} +const { storage }: Props = Astro.props; +const scopes = ['session', 'agent', 'user', 'store']; +--- + + +
+ SCOPE OF READ +
+ { + scopes.map((scope, index) => ( + + )) + } +
+
+
+
+ 01

Short-term rows

Recent conversation window · relational records

+ +
+
+ 02

Rolling digest

Optional summary · narrative continuity

+ +
+
+ 03

Long-term vector recall

Semantic facts · relevance across sessions

+ +
+
+

{storage}

+
+ + + + diff --git a/src/components/OpenSourceShowcase.astro b/src/components/OpenSourceShowcase.astro index 3b62a423..d34bab6f 100644 --- a/src/components/OpenSourceShowcase.astro +++ b/src/components/OpenSourceShowcase.astro @@ -1,5 +1,6 @@ --- import { getCollection } from 'astro:content'; +import { isExternalHref } from '../utils/external-links.mjs'; import KaosGraph from './KaosGraph.astro'; interface Action { @@ -77,7 +78,7 @@ const statline = (id: string) => repoMetrics[id].map(({ value, label }: { value:

{projects.kompute.text}

{projects.kompute.donationText}{' '} - {projects.kompute.donationLinkText}. + {projects.kompute.donationLinkText}.

{ @@ -95,7 +96,7 @@ const statline = (id: string) => repoMetrics[id].map(({ value, label }: { value:
{ projects.kompute.actions.map(({ label, href, kind }) => ( - + {label} )) @@ -153,8 +154,8 @@ const statline = (id: string) => repoMetrics[id].map(({ value, label }: { value:

{projects.ecosystem.text}

{statline('ecosystem').map((stat) => {stat})}
@@ -183,7 +184,8 @@ const statline = (id: string) => repoMetrics[id].map(({ value, label }: { value: {label} diff --git a/src/components/PartnerDirectory.astro b/src/components/PartnerDirectory.astro index 9cca36cf..27eb80e7 100644 --- a/src/components/PartnerDirectory.astro +++ b/src/components/PartnerDirectory.astro @@ -35,7 +35,7 @@ const entries = partners.map(({ data: partner }) => ({ {partner.links.length > 0 && (