Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions .github/workflows/sync-skills.yml
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,49 @@ jobs:
echo "- orphan pruning" >> /tmp/changed-components.txt
fi

- name: Backfill compliance artifacts from compliance.d
# Sourced skills are rsynced with --delete, so a skill-card.md committed
# into a catalog dir is deleted on the next run. Cards for sourced skills
# therefore live in compliance.d/<catalog_path>/ — outside every rsync
# target and every prune root — and are copied in AFTER the rsync.
#
# Upstream ALWAYS wins: we only copy where the file is absent once the
# rsync has finished. The moment a source repo ships its own card, that
# card lands via rsync and this step becomes a no-op for it — at which
# point the compliance.d entry is dead weight and should be deleted.
# Both states are reported in the PR body so the debt stays visible.
#
# NOTE: skill-card.md only. skill.oms.sig is deliberately NOT backfilled
# — a signature is an attestation over specific bytes, and copying one
# over content it was not generated from would publish an artifact that
# fails verification. Signing stays a per-source-repo nvskills-ci task.
run: |
set -euo pipefail
: > /tmp/overlay-backfilled.txt
: > /tmp/overlay-retirable.txt
[ -d compliance.d ] || exit 0

while IFS= read -r card; do
dest="${card#compliance.d/}"
skill_dir=$(dirname "$dest")
if [ ! -d "$skill_dir" ]; then
echo " ⚠ $skill_dir absent from catalog — overlay entry has no target"
continue
fi
if [ -f "$dest" ]; then
echo " ✓ $skill_dir — upstream now ships a card; retire compliance.d entry"
echo "$skill_dir" >> /tmp/overlay-retirable.txt
else
cp "$card" "$dest"
echo " ↪ $skill_dir — backfilled card from compliance.d"
echo "$skill_dir" >> /tmp/overlay-backfilled.txt
fi
done < <(find compliance.d -type f -name skill-card.md)

if [ -s /tmp/overlay-backfilled.txt ]; then
echo "- compliance backfill" >> /tmp/changed-components.txt
fi

- name: Rebuild plugin payload
# Regenerate plugins/bionemo-agent-toolkit/ so the sync PR ships skills +
# payload together and stays plugin-sync green. (build-plugins.py fork is
Expand Down Expand Up @@ -156,6 +199,16 @@ jobs:
echo "**Failed to sync:**"
cat /tmp/failed-components.txt
fi
if [ -s /tmp/overlay-backfilled.txt ]; then
echo ""
echo "**Cards backfilled from \`compliance.d\` (upstream still owes a \`skill-card.md\`):**"
while read -r d; do echo "- \`$d\`"; done < /tmp/overlay-backfilled.txt
fi
if [ -s /tmp/overlay-retirable.txt ]; then
echo ""
echo "**Upstream now ships its own card — delete these \`compliance.d\` entries:**"
while read -r d; do echo "- \`compliance.d/$d/skill-card.md\`"; done < /tmp/overlay-retirable.txt
fi
echo ""
echo "Sourced content changes + any pruning are shown in the diff. Native"
echo "skills (NIM, workflows, parabricks, genomics, cuEquivariance) are untouched."
Expand Down
52 changes: 52 additions & 0 deletions compliance.d/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
<!-- SPDX-License-Identifier: CC-BY-4.0 -->
# `compliance.d/` — compliance artifacts for sourced skills

Skills vendored from a source repo are rsynced with `--delete` (see
[`../.github/workflows/sync-skills.yml`](../.github/workflows/sync-skills.yml)),
so **a file committed directly into a sourced catalog dir is deleted on the next
nightly sync.** That is what happened to the KERMT evals (see
[`../docs/sync-findings.md`](../docs/sync-findings.md)).

This directory holds `skill-card.md` files for sourced skills, mirroring the
catalog path, outside every rsync target and every prune root:

```
compliance.d/<catalog_dir>/<skill>/skill-card.md
↓ backfilled after the rsync, only if absent
<catalog_dir>/<skill>/skill-card.md
```

## Upstream always wins

The **Backfill compliance artifacts** step in the sync workflow copies a card
into the catalog *only when the rsync left no card there*. As soon as a source
repo ships its own `skill-card.md`, that card lands via rsync and the backfill
becomes a no-op for that skill — no flag to flip, no coordination needed.

The sync PR body reports both states:

- **backfilled** — upstream still owes a card; this is the outstanding debt list
- **retirable** — upstream now ships its own card, so the `compliance.d` entry
here is dead weight and should be deleted in that PR

## What does *not* belong here

**`skill.oms.sig`.** A signature is a cryptographic attestation over specific
bytes. Copying one over content it was not generated from produces an artifact
that fails verification — and `NVIDIA/skills`' sync detects exactly this case
and reverts the skill. Signing stays a per-source-repo task, performed by
commenting `/nvskills-ci` on a PR in the repo that owns the skill.

**Evals.** Eval definitions must be co-located with the skill in its source repo
(`SRC-10`), for the same `--delete` reason.

## Current entries

| Catalog path | Source repo | Upstream card PR |
|---|---|---|
| `open-models-skills/kermt/*` (8) | `NVIDIA-BioNeMo/KERMT` | [KERMT#26](https://github.com/NVIDIA-BioNeMo/KERMT/pull/26) |
| `open-models-skills/proteina-complexa/*` (5) | `NVIDIA-BioNeMo/Proteina-Complexa` | [Proteina-Complexa#60](https://github.com/NVIDIA-BioNeMo/Proteina-Complexa/pull/60) |
| `library-skills/nvMolKit` (1) | `NVIDIA-BioNeMo/nvMolKit` | [nvMolKit#248](https://github.com/NVIDIA-BioNeMo/nvMolKit/pull/248) |

The same card content was opened as a PR against each source repo. When those
merge, the corresponding entries here become retirable.
71 changes: 71 additions & 0 deletions compliance.d/library-skills/nvMolKit/skill-card.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
## Description: <br>
Guides an agent to write correct code against the installed nvMolKit Python API for GPU-accelerated, batched RDKit-style cheminformatics — Morgan fingerprints, Tanimoto/cosine similarity, ETKDG conformer embedding, MMFF/UFF optimization, TFD, conformer RMSD, Butina clustering, and substructure search. <br>

This skill is ready for commercial/non-commercial use. <br>

## Owner
NVIDIA (Kevin Boyd, @scal444) <br>

### License/Terms of Use: <br>
Apache-2.0 <br>

## Use Case: <br>
Cheminformaticians and ML engineers importing `nvmolkit.*`, debugging an nvMolKit call, deciding between nvMolKit and RDKit for a batched workflow, or wiring nvMolKit results into a torch/numpy pipeline. Out of scope: building nvMolKit from source. <br>

### Requirements/Dependencies: <br>
Requires API Key or External Credential: No <br>
Credential Type(s): None <br>

* An existing nvMolKit installation (`uv pip install --torch-backend=cu128 nvmolkit`) <br>
* CUDA-capable NVIDIA GPU <br>
* Python with RDKit available for interoperation <br>

Do not include secrets in prompts/logs/output; use least-privilege credentials; rotate keys as appropriate. <br>

### Deployment Geography for Use: <br>
Global <br>

## Known Risks and Mitigations: <br>
Risk: The skill emits code for the agent or user to execute; incorrect API usage could produce silently wrong cheminformatics results — for example mismatched fingerprint parameters yielding similarity scores that are not comparable across runs. <br>
Mitigation: The skill documents result types and the asynchronous execution model (`AsyncGpuResult`, `Device3DResult`) explicitly, and includes a verification step to run against the install before writing real code. Users should review generated code before executing it. <br>

Risk: nvMolKit and RDKit can differ numerically for the same nominal operation (conformer generation and force-field optimization are stochastic and hardware-sensitive), so results may not reproduce bit-for-bit across backends. <br>
Mitigation: The skill covers where nvMolKit is and is not an appropriate substitute for RDKit, so users choose the backend deliberately rather than assuming equivalence. <br>

Risk: Batched GPU operations on large molecule libraries can exhaust GPU memory mid-run. <br>
Mitigation: The skill documents `HardwareOptions` configuration for batch and device control. <br>

Risk: Asynchronous result handles can be read before completion if the execution model is misunderstood, yielding empty or partial data. <br>
Mitigation: The skill's result-type documentation is explicit about when a result must be awaited or materialized. <br>

## Reference(s): <br>
- [nvMolKit repository](https://github.com/NVIDIA-BioNeMo/nvMolKit) <br>
- [RDKit documentation](https://www.rdkit.org/docs/) — the API surface nvMolKit mirrors <br>
- `SKILL.md` in this skill directory — result types, `HardwareOptions` / `SubstructSearchConfig`, and worked recipes <br>

## Skill Output: <br>
**Output Type(s):** [Code, Analysis] <br>
**Output Format:** [Markdown with inline Python code blocks] <br>
**Output Parameters:** [1D] <br>
**Other Properties Related to Output:** [The skill produces code and guidance; it does not itself execute cheminformatics workloads. Generated code should be reviewed before execution.] <br>

## Evaluation Agents Used: <br>
Target agents: `claude-code`, `codex`. NVSkills-Eval has not yet been run against this skill — see Evaluation Results. <br>

## Evaluation Tasks: <br>
11 functional evaluation tasks in `evals/evals.json` covering fingerprinting, similarity, conformer generation, optimization, clustering, and substructure search, plus 2 trigger-activation cases in `evals/trigger_evals.json`. <br>

## Evaluation Metrics Used: <br>
Planned NVSkills-Eval dimensions: Security, Correctness, Discoverability, Effectiveness, Efficiency. <br>

## Evaluation Results: <br>
Pending. NVSkills-Eval has not been run for this skill; results and a `BENCHMARK.md` will be published when the evaluation pipeline runs. <br>

## Skill Version(s): <br>
ea68428 (source: git SHA, committed 2026-08-03) <br>

## Ethical Considerations: <br>
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal team to ensure this skill meets requirements for the relevant industry and use case and addresses unforeseen product misuse. <br>

(For Release on NVIDIA Platforms Only) <br>
Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/). <br>
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
## Description: <br>
Converts a grover_base checkpoint into a hybrid checkpoint by adding a randomly-initialized cMIM decoder and latent distribution, then continues pretraining on the user's corpus in hybrid (vocab + contrast) mode. <br>

This skill is ready for commercial/non-commercial use. <br>

## Owner
NVIDIA (evax@nvidia.com) <br>

### License/Terms of Use: <br>
Apache-2.0 <br>

## Use Case: <br>
ML research engineers who want the cMIM contrastive objective on top of an existing grover_base KERMT checkpoint without retraining from scratch. Functionally `kermt-continue-pretrain` with a one-time checkpoint-conversion step prepended. <br>

### Requirements/Dependencies: <br>
Requires API Key or External Credential: Optional <br>
Credential Type(s): API key — `WANDB_API_KEY` for optional Weights & Biases run tracking <br>

* `kermt-setup` completed (supplies the `kermt:latest` image) <br>
* Docker, NVIDIA Container Toolkit, CUDA-capable NVIDIA GPU (multi-GPU supported via DDP) <br>
* A grover_base checkpoint (encoder-only, or encoder + vocab heads) <br>
* A pretraining corpus CSV <br>

Do not include secrets in prompts/logs/output; use least-privilege credentials; rotate keys as appropriate. <br>

### Deployment Geography for Use: <br>
Global <br>

## Known Risks and Mitigations: <br>
Risk: The newly added cMIM decoder and latent distribution are randomly initialized, so the converted checkpoint performs worse than its source until sufficient hybrid pretraining has run — a checkpoint taken too early is silently degraded. <br>
Mitigation: The conversion step is explicitly separated from the training step, and `kermt-monitor` exposes validation loss so users can confirm convergence before adopting the result. <br>

Risk: Continued pretraining is a long-running, multi-GPU workload that a single agent instruction can start, potentially consuming days of GPU time. <br>
Mitigation: Runs launch detached with a run manifest; `kermt-monitor` provides progress visibility and the container identifiers needed to terminate early. <br>

Risk: Supplying a checkpoint that is not grover_base (e.g. already cmim or hybrid) would produce an invalid conversion. <br>
Mitigation: The skill validates the source checkpoint type before conversion. <br>

Risk: Conversion and data preparation write new checkpoint and shard/vocab/feature artifacts that can overwrite prior output. <br>
Mitigation: Artifacts are written under an explicit run/output path; the source checkpoint is not modified in place. <br>

Risk: When Weights & Biases tracking is enabled, run metadata is transmitted to a third-party service. <br>
Mitigation: W&B tracking is optional and off unless the user supplies `WANDB_API_KEY`. <br>

## Reference(s): <br>
- [KERMT repository](https://github.com/NVIDIA-BioNeMo/KERMT) <br>
- `agent/scripts/run_pretrain_local.py` — extended usage examples <br>
- Related skills: `kermt-setup`, `kermt-monitor`, `kermt-continue-pretrain`, `kermt-pretrain-scratch` <br>

## Skill Output: <br>
**Output Type(s):** [Files, Analysis] <br>
**Output Format:** [Converted hybrid checkpoint; subsequent training checkpoints; shard/vocab/feature artifacts; training logs; `run.json` manifest; Markdown launch summary] <br>
**Output Parameters:** [1D — run identifier, container id, converted checkpoint path, output paths] <br>
**Other Properties Related to Output:** [Detached execution: the skill returns after launch, not after training completes.] <br>

## Evaluation Agents Used: <br>
Target agents: `claude-code`, `codex`. NVSkills-Eval has not yet been run against this skill — see Evaluation Results. <br>

## Evaluation Tasks: <br>
4 evaluation tasks defined in `evals/evals.json`, covering source-checkpoint validation, cMIM conversion, corpus preparation, and detached launch. <br>

## Evaluation Metrics Used: <br>
Planned NVSkills-Eval dimensions: Security, Correctness, Discoverability, Effectiveness, Efficiency. <br>

## Evaluation Results: <br>
Pending. NVSkills-Eval has not been run for this skill; results and a `BENCHMARK.md` will be published when the evaluation pipeline runs. <br>

## Skill Version(s): <br>
b1c082c (source: git SHA, committed 2026-07-17) <br>

## Ethical Considerations: <br>
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal team to ensure this skill meets requirements for the relevant industry and use case and addresses unforeseen product misuse. <br>

Models produced by this skill inherit the composition and biases of the user's pretraining corpus. Downstream predictions are research hypotheses and must not be used as the sole basis for clinical, safety, or regulatory decisions. <br>

(For Release on NVIDIA Platforms Only) <br>
Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/). <br>
Loading