diff --git a/.agents/rules/coding-standards.md b/.agents/rules/coding-standards.md index 3b1738b0..0d07f84e 100644 --- a/.agents/rules/coding-standards.md +++ b/.agents/rules/coding-standards.md @@ -19,8 +19,8 @@ These rules are universally applied across all aspects of this project. **You MU 10. **URL Validation**: After generating any URLs, always test whether the URL is valid by verifying it directly (e.g., curling). ## 2. Environment and Dependency Management -- **Environment Isolation**: You must use the appropriate isolated Conda environment depending on the module or skill being developed (see `@.agents/rules/mcp-environments.md`). The `base-agent` environment serves as the generic fallback for standard processing. -- **Dependency Versions**: Never globally force arbitrary PyTorch or library versions (e.g., do not globally demand `torch 2.9.1`). PyTorch and critical packages are rigorously managed per-environment by their respective `core_env.yaml` schemas in `conda-envs/*`. +- **Environment Isolation**: You must use the appropriate environment via the `venv/run` launcher depending on the task: a shared project (`cpu`, `mlip`, `fairchem`) or the research stack a skill declares in `metadata.venv` (see `.agents/rules/mcp-environments.md`). The `cpu` environment serves as the generic fallback for standard processing without torch. +- **Dependency Versions**: Dependencies are rigorously managed per uv project in `venv//pyproject.toml` and committed `uv.lock`. Never arbitrarily force or install package versions globally. Optional extras (`openmm`, `docking`, `pymol`, `void`, `transport`) handle system-dependent packages. - **Failures**: NEVER implement fallback functions when package imports fail. Debug the root cause and fix the environment configuration. diff --git a/.agents/rules/mcp-environments.md b/.agents/rules/mcp-environments.md index 666293b1..637ee58d 100644 --- a/.agents/rules/mcp-environments.md +++ b/.agents/rules/mcp-environments.md @@ -2,42 +2,81 @@ trigger: always_on --- -# MCP Server Environment Rules - -When running scripts or debugging code related to specific MCP servers, you MUST use the corresponding Conda environment defined in `mcp_config.json`. - -For general development logic that cuts across tools, usually `base-agent` is sufficient, but when importing specific libraries that are isolated (like `atomate2`, `jobflow_remote`, `mace`, `fairchem`), you must use the correct environment. - -## Installation Instructions - -For detailed installation instructions, please refer to the `README.md` and `install.sh` located in each environment's directory under `conda-envs//`. - -- **base-agent**: `conda-envs/base-agent/` (Core: pymatgen, ase, rdkit, packmol) -- **mace-agent**: `conda-envs/mace-agent/` (Core: mace-torch, pymatgen, ase) -- **matgl-agent**: `conda-envs/matgl-agent/` (Core: matgl, pymatgen, dgl) -- **fairchem-agent**: `conda-envs/fairchem-agent/` (Core: fairchem-core≥2.18, pymatgen, Python 3.12) -- **atomate2-agent**: `conda-envs/atomate2-agent/` (Core: atomate2, pymatgen) -- **adit-agent**: `conda-envs/adit-agent/` (Core: ADiT, lightning, hydra, PyG) -- **diffcsp-agent**: `conda-envs/diffcsp-agent/` (Core: DiffCSP++, hydra, PyG, pyxtal) -- **mattergen-agent**: `conda-envs/mattergen-agent/` (Core: MatterGen, PyG, lightning) -- **smol-agent**: `conda-envs/smol-agent/` (Core: smol, pymatgen) -- **drugdisc-agent**: `conda-envs/drugdisc-agent/` (Core: rdkit, autodock-vina, pdbfixer, meeko) -- **xrd-agent**: `conda-envs/xrd-agent/` (Core: DARA, pymatgen) -- **orca-agent**: `conda-envs/orca-agent/` (Core: scine_utilities, scine_readuct, ase). Requires `ORCA_BINARY_PATH` environment variable pointing to the ORCA binary. No MCP server, scripts only. -- **phasefield-agent**: `conda-envs/phasefield-agent/` (Core: fipy, scipy, numpy, imageio). No MCP server, scripts only. -- **calphad-agent**: `conda-envs/calphad-agent/` (Core: pycalphad, pymatgen). No MCP server, scripts only. -- **nmr-agent**: `conda-envs/nmr-agent/` (Core: nmrsim, nmrglue, pot). No MCP server, scripts only. -- **react-ot-agent**: `conda-envs/react-ot-agent/` (Core: PyTorch). No MCP server, scripts only. - -| MCP Server | Conda Environment | Python Path | +# MCP Server and Environment Runtime Rules + +AtomisticSkills runs all skill scripts and MCP servers through the unified launcher `venv/run`. The runtime uses three shared `uv` projects under `venv/` plus one pinned project per research stack, with pre-built container images as an automatic fallback when host requirements are not met. + +## 1. Environments Overview + +| Environment | Primary Packages | GPU | MCP Servers | +| :--- | :--- | :--- | :--- | +| `cpu` | ASE, pymatgen, RDKit, pycalphad, MDAnalysis (no torch) | No | `base`, `atomate2`, `drugdisc`, `smol` | +| `mlip` | PyTorch 2.14.1, MACE-torch 0.3.16, MatGL 4.1.0, nvalchemi-toolkit 0.2.0 | Yes | `mace`, `matgl` | +| `fairchem` | PyTorch 2.13.0, fairchem-core 2.23.0, nvalchemi-toolkit 0.2.0 | Yes | `fairchem` | + +### Research Stacks + +Pinned to the versions they were verified with, rather than tracking the latest releases: + +| Environment | Used by | Platforms | | :--- | :--- | :--- | -| matgl | `matgl-agent` | `/envs/matgl-agent/bin/python` | -| mace | `mace-agent` | `/envs/mace-agent/bin/python` | -| fairchem | `fairchem-agent` | `/envs/fairchem-agent/bin/python` | -| materials_tools | `base-agent` | `/envs/base-agent/bin/python` | -| atomate2 | `atomate2-agent` | `/envs/atomate2-agent/bin/python` | -| smol | `smol-agent` | `/envs/smol-agent/bin/python` | -| adit | `adit-agent` | `/envs/adit-agent/bin/python` | -| diffcsp | `diffcsp-agent` | `/envs/diffcsp-agent/bin/python` | -| mattergen | `mattergen-agent` | `/envs/mattergen-agent/bin/python` | -| drugdisc | `drugdisc-agent` | `/envs/drugdisc-agent/bin/python` | +| `adit`, `diffcsp`, `mattergen` | the generative MCP servers and skills | x86_64 natively with glibc ≥ 2.32 (CUDA 12.6 or 13 by driver); elsewhere the `generative` image | +| `msms` | `chem-msms-predict` (ICEBERG 2.1, CPU) | x86_64 only, glibc ≥ 2.31 and GCC 9's libstdc++ | +| `reactot` | `chem-react-ot` | x86_64 and aarch64 | +| `scd` | `ml-property-predict-scd` | x86_64 only, glibc ≥ 2.32 | + +### Why Three Shared Environments? +1. **Package conflicts**: `mace-torch` pins `e3nn==0.4.4`, whereas `fairchem-core` requires `e3nn>=0.5`. +2. **PyTorch versions**: `fairchem-core` 2.23 requires `torch~=2.13`, while `mlip` runs PyTorch 2.14.1. +3. **NumPy constraints**: `nvalchemi-toolkit` 0.2 requires `numpy<2.4`, so GPU environments use NumPy 2.3.5 while `cpu` uses NumPy 2.5+. + +### Optional Extras +Environments support optional extras specified as `+` (e.g., `cpu+openmm`, `cpu+docking`): +- `openmm`: OpenMM and PDBFixer (requires glibc ≥ 2.34). +- `pymol`: PyMOL open-source (x86_64 only). +- `docking`: AutoDock Vina (builds against Boost on aarch64). +- `void`: VOID guest docking (installed from git). +- `transport`: AMSET and BoltzTraP2 (requires git and cmake). + +## 2. Server Runtime Mapping + +| MCP Server | venv | Image | GPU | Description | +| :--- | :--- | :--- | :--- | :--- | +| `base` | `cpu` | `cpu` | No | Materials Project query, structure manipulation, literature | +| `atomate2` | `cpu` | `cpu` | No | Atomate2 and Jobflow calculation workflows | +| `drugdisc` | `cpu` | `cpu` | No | Small-molecule standardization, descriptors, fingerprints | +| `smol` | `cpu` | `cpu` | No | Cluster expansion and Monte Carlo simulations | +| `mace` | `mlip` | `mlip` | Yes | MACE foundation models (relax, MD, features) | +| `matgl` | `mlip` | `mlip` | Yes | MatGL models (CHGNet, TensorNet, M3GNet) | +| `fairchem` | `fairchem` | `fairchem` | Yes | FairChem models (UMA, eSEN) | +| `adit` | `adit` | `generative` | Yes | ADiT all-atom diffusion transformer | +| `diffcsp` | `diffcsp` | `generative` | Yes | DiffCSP++ crystal structure generation | +| `mattergen` | `mattergen` | `generative` | Yes | MatterGen generative diffusion model | + +The generative servers run from their uv projects on x86_64 hosts with glibc ≥ 2.32 (PyG's wheels need it); elsewhere `venv/run` uses the `generative` container image (linux/amd64 and linux/arm64; on arm64 it compiles the PyG extensions from source, since PyG publishes no aarch64 wheels). + +## 3. No Conda + +Since 2.0.0 nothing runs from conda. Stacks that cannot share an environment get their own pinned uv project (the research stacks above); host builds use a uv extra (`mlip+lammps` provides what `mat-lammps-md` needs to compile LAMMPS against the `mlip` environment, `fairchem+lammps` the LAMMPS wheel and `fairchem-lammps`). + +## 4. Runtime Selection and Launcher Backend + +All skills and servers execute through the `venv/run` launcher: +- `venv/run [+] [args...]` +- `venv/run --server ` +- `venv/run --setup` (prepares environments) +- `venv/run --doctor` (checks runtime status and diagnostics) + +The runtime backend is controlled by `ATOMISTIC_RUNTIME` (or `~/.config/atomistic_skills.yaml`): +- `auto` (default): Uses native `uv` if the machine is Linux x86_64/aarch64 with compatible glibc (per `venv/platforms.tsv`) and a C compiler (`gcc`). Otherwise, automatically falls back to container images (`ghcr.io/learningmatter-mit/atomisticskills-:2.0.0`). +- `uv`: Enforces host `uv` execution. +- `docker`, `podman`, `apptainer`, `singularity`: Enforces container execution with host paths mounted at identical locations. + +## 5. Shell CLI Fallback for MCP Tools + +When an MCP server is not connected via stdio, any MCP tool can be executed directly from the shell: +```bash +venv/run python -m src.mcp_server.cli key=value ... +``` +- Pass `--list` to view all available tools for a server. +- Multiple tool calls chained in one command share process memory (e.g., `load_model` followed by `relax_structure`). diff --git a/.agents/rules/release-standards.md b/.agents/rules/release-standards.md index a21de030..859400a3 100644 --- a/.agents/rules/release-standards.md +++ b/.agents/rules/release-standards.md @@ -17,13 +17,43 @@ Use [Semantic Versioning](https://semver.org/): `vMAJOR.MINOR.PATCH` ## Pre-Tag Checklist -1. Rebuild the doc site to get accurate public counts: +0. Bump the version and propagate it to every published manifest. `VERSION` at + the repository root is the single source; `.claude-plugin/plugin.json`, + `.claude-plugin/marketplace.json` and `server.json` are derived from it. + The tag must match `VERSION` exactly. ```bash - conda run -n base-agent python site/build_skills.py + venv/run cpu python tools/sync_version.py --set 1.x.y + venv/run cpu python tools/sync_version.py --check ``` -2. Read counts from `site/skills_index.js` and git (source of truth): + > CI runs `--check` on every push, so a tag can never ship with the four + > files disagreeing. + + If `docker/images.json` changed in this release, re-render the plugin's MCP + wiring as well, since the image tags follow the version: + ```bash + python docker/render.py plugin-mcp + ``` + +1. Verify derived files, platform locks, and migrations: + ```bash + python docker/render.py servers --check + python docker/render.py plugin-mcp --check + venv/run cpu python tools/lock_platforms.py --check + venv/run cpu python tools/migrate_skill_commands.py --check + ``` + +2. Run the test suites: + ```bash + venv/run cpu python -m pytest tests/test_launcher.py tests/test_skill_runtime.py tests/test_uv_projects.py tests/test_images_and_manifests.py tests/test_tool_cli.py -q + ``` + +3. Rebuild the doc site to get accurate public counts: + ```bash + venv/run cpu python site/build_skills.py + ``` +4. Read counts from `site/skills_index.js` and git (source of truth): ```bash - # skills, tools, servers (= conda-env dirs) — from rebuilt index + # skills, tools, servers — from rebuilt index python3 -c " import re content = open('site/skills_index.js').read() @@ -37,8 +67,8 @@ Use [Semantic Versioning](https://semver.org/): `vMAJOR.MINOR.PATCH` ``` > **Note**: Workflows live as plain `*.md` files in `.agents/workflows/` (e.g. `drug-hit-finding-htvs.md`), **not** as `WORKFLOW.md`. Always use `ls .agents/workflows/*.md | wc -l` for the current count and `git ls-tree -r --name-only | grep "workflows/.*\.md" | wc -l` for the previous-tag count. -3. Ensure all pre-commit hooks pass on HEAD. -4. Confirm the MCP server does not expose built-in agent tools (e.g. `task_boundary`, `notify_user`). +5. Ensure all pre-commit hooks pass on HEAD. +6. Confirm the MCP server does not expose built-in agent tools (e.g. `task_boundary`, `notify_user`). ## Tag Message Format diff --git a/.agents/rules/skill-standards.md b/.agents/rules/skill-standards.md index b0ad94bd..ab3c66cf 100644 --- a/.agents/rules/skill-standards.md +++ b/.agents/rules/skill-standards.md @@ -1,17 +1,17 @@ --- trigger: model_decision -description: Rules to implement a skill under `.agents/skills/` +description: Rules to implement a skill under `skills/` --- # Skill Standards -All modular capabilities in this project should be implemented as "Skills" within the `.agents/skills/` directory. This rule ensures consistency, discoverability, and reusability for the agent. +All modular capabilities in this project should be implemented as "Skills" within the `skills/` directory. This rule ensures consistency, discoverability, and reusability for the agent. ## Directory Structure Each skill must reside in its own subdirectory with the following structure: ``` -.agents/skills// +skills// ├── SKILL.md # Required: Main documentation ├── scripts/ # Optional: Helper scripts │ ├── script1.py @@ -35,19 +35,24 @@ The `SKILL.md` file must follow this standardized structure: --- name: skill-name-in-kebab-case description: Concise one-sentence summary of the skill's purpose and outcome. -category: category-name +metadata: + category: [category-name] + venv: [cpu] # or mlip, fairchem, or a research stack (adit, diffcsp, mattergen, msms, reactot, scd) --- ``` +**Only the six keys of the [Agent Skills](https://agentskills.io) spec are allowed at the top level** — `name`, `description`, `license`, `compatibility`, `metadata`, `allowed-tools`. Claude Code tolerates extra keys, but claude.ai skill uploads and the Skills API reject them with a hard error, which would break importing this repository into Claude Science. Project-specific fields therefore go inside the free-form `metadata` map. + **Guidelines:** - `name`: Use lowercase letters, numbers, and hyphens only (kebab-case) - `description`: Should be clear enough for the agent to decide if this skill is relevant to a user query. **The description must state what the skill is used for, NOT how the skill works.** Avoid mid-sentence colons (`: `) in unquoted values — they break YAML parsing. -- `category`: Must be one of the following (use a YAML list like `[materials, chemistry]` if multiple apply): +- `metadata.category`: Always a YAML list, even for a single value (e.g. `[materials]` or `[materials, chemistry]`). Must be drawn from: - `materials`: Materials science simulation and analysis skills (prefix: `mat-`) - `chemistry`: Chemistry and molecular simulation skills (prefix: `chem-`) - `machine-learning`: MLIP training, model selection, and ML workflows (prefix: `ml-`) - `drug-discovery`: Drug design, docking, and molecular property prediction (prefix: `drug-`) - `general`: General-purpose research utilities (prefix: `general-`) +- `metadata.venv`: A YAML list declaring the uv environments used: the shared projects (`[cpu]`, `[mlip]`, `[fairchem]`) or a research stack's own project under `venv/` (e.g. `[msms]`). A new stack that cannot share an environment gets its own pinned uv project rather than a conda environment. ### 2. Title and Goal Section Begin with a level-1 header matching the skill name, followed by a `## Goal` section: @@ -55,6 +60,18 @@ Begin with a level-1 header matching the skill name, followed by a `## Goal` sec ```markdown # Skill Name + +> [!NOTE] +> Steps written `server.tool` are MCP tool calls: `server.tool` is the `tool` +> tool of the `server` server (`mcp____`, or +> `mcp__plugin_atomistic-skills___` when installed as a plugin). +> Without a connected server, run the same tools from the shell. Tools named in +> one command share a process, so a model loaded by `load_model` stays loaded: +> +> ```bash +> ${CLAUDE_SKILL_DIR}/../../venv/run python -m src.mcp_server.cli key=value +> ``` + ## Goal Clearly state what this skill achieves. Use precise technical language and, when applicable, include mathematical notation (e.g., "To determine the thermodynamic melting temperature ($T_m$) of a bulk material"). ``` @@ -62,22 +79,29 @@ Clearly state what this skill achieves. Use precise technical language and, when ### 3. Instructions Section Provide numbered, step-by-step instructions. Each step should: - **State the objective clearly** (e.g., "Background Research", "Phase Preparation") -- **Provide specific commands** with environment annotations +- **Provide specific commands** with the launcher invocation - **Include all necessary parameters** with explanations - **Link to related skills** when appropriate **Format for code blocks:** +Skill commands use the self-locating launcher form: ````markdown ```bash -# Env: -python .agents/skills//scripts/ + + + + + + + + +
+
+
Home / Documentation
+

Remote Cluster: Atomate2 & JobFlow Remote Setup

+
+
+
+
+
+

AtomisticSkills — Open-sourced AI research infrastructure

+ + + diff --git a/site/build_skills.py b/site/build_skills.py index 2ec4ccea..a64ecce2 100644 --- a/site/build_skills.py +++ b/site/build_skills.py @@ -1,7 +1,7 @@ #!/usr/bin/env python3 """ Build static skill subpages for AtomisticSkills docs. -Reads SKILL.md + example READMEs from .agents/skills/, writes HTML to site/skills/. +Reads SKILL.md + example READMEs from skills/, writes HTML to site/skills/. Run from the project root: python site/build_skills.py """ @@ -16,7 +16,7 @@ # Fix: Define PROJECT_ROOT relative to this file's location (site/build_skills.py) PROJECT_ROOT = Path(__file__).resolve().parent.parent -SKILLS_SRC_DIR = PROJECT_ROOT / ".agents" / "skills" +SKILLS_SRC_DIR = PROJECT_ROOT / "skills" DOCS_SRC_DIR = PROJECT_ROOT / "docs" WORKFLOWS_SRC_DIR = PROJECT_ROOT / ".agents" / "workflows" SITE_DIR = PROJECT_ROOT / "site" @@ -84,6 +84,10 @@ def parse_frontmatter(content): except Exception as e: print(f"Error parsing frontmatter: {e}") content = content[end + 3 :].strip() + # `category` lives under the spec-allowed `metadata` map (the Agent Skills + # spec rejects unknown top-level keys). Hoist it and normalise to a list. + cats = (meta.get("metadata") or {}).get("category", meta.get("category", [])) + meta["category"] = cats if isinstance(cats, list) else [cats] return meta, content @@ -394,7 +398,7 @@ def make_skill_page( ← Back to Home
- View on GitHub + View on GitHub
@@ -416,7 +420,7 @@ def make_skill_page( {examples_html}
- + View this Skill on GitHub @@ -922,10 +926,14 @@ def build_skills(): servers_count = 0 server_tools_count = {} - # Count conda envs - conda_envs_dir = PROJECT_ROOT / "conda-envs" - if conda_envs_dir.exists(): - servers_count = len([d for d in conda_envs_dir.iterdir() if d.is_dir()]) + # Count MCP servers (venv/servers.tsv, rendered from docker/images.json) + servers_table = PROJECT_ROOT / "venv" / "servers.tsv" + if servers_table.exists(): + servers_count = sum( + 1 + for line in servers_table.read_text().splitlines() + if line and not line.startswith("#") + ) mcp_dir = PROJECT_ROOT / "src" / "mcp_server" if mcp_dir.exists(): diff --git a/site/developer_guide.html b/site/developer_guide.html index 3c860c2c..04de2dfa 100644 --- a/site/developer_guide.html +++ b/site/developer_guide.html @@ -82,7 +82,7 @@

Developer Guide

AtomisticSkills — Open-sourced AI research infrastructure