microscope-agent

A minimal MCP server for controlling a real Nikon Ti2-E microscope + Baumer GenICam camera from an AI agent (Claude Desktop, Claude Code, or any other MCP client).

Tools

Exactly 5, by design - kept deliberately minimal:

• get_pos(backend) - current stage (x, y, z) position, in microns. A cheap, on-demand sync primitive, not something to call before every move (get_image/move already return position as part of their result).
• move(x, y, backend, confirm) - move the XY stage to an absolute position, returns the actual resulting position. XY-only, on purpose (no absolute Z move is reachable from chat - crash risk into the sample).
• get_image(confirm, exposure_time_us, gain, crop, max_dimension, backend) - capture a frame from the Baumer camera (backend="sdk", the default) or a simulated one from the mock (backend="mock", no hardware, no confirm needed - set CONFOCAL_MOCK_FRAME_PATH to choose the PNG it serves), paired with the exact stage position it was taken at. Returns both the metadata and an embedded image preview, so the model can actually see the picture, not just a file path. crop (optional {"x","y","width","height"} fractions of the full frame) restricts the embedded preview to a region of interest instead of resending the whole frame; the full-res file on disk is always uncropped. max_dimension overrides the preview's default long-edge cap for that call.

get_pos/move/get_image all return stage_revision, a monotonic counter bumped whenever a call observes the stage at a different position than the last one this process saw - so a model holding an older result can tell "has the stage moved since then" (e.g. someone touched the joystick) without diffing raw coordinates itself. get_image additionally returns frame_id, a per-process counter identifying that specific capture.

• get_move_history(limit) - every point move() has actually sent the stage to this session, so "where have we already been" doesn't need to be re-derived from conversation history.

• estop(action) - emergency stop. "engage" forbids all stage motion for every process on the machine via a flag file; "status" reports it. Needs no confirm: a stop is always safe. Release is deliberately not reachable from chat - a human clears it with python -m acquisition.estop release.

backend is "mock" (default, safe, simulated) or "sdk" (real hardware - move()/get_image() require confirm=True for anything that touches real hardware). See mcp_server/loop_tools.py's header comment for the full design rationale (why no get_time(), why no stage_revision yet, etc.).

No NIS-Elements involved anywhere - the camera is reached directly via GenICam/GenTL, and the stage via the Ti2 ActiveX SDK, both independent of whether NIS-Elements software is even running.

Adaptive time-lapse (timelapse/)

Not an MCP tool - a loop that sits beside the tools and calls get_image() itself. It captures on a slow interval, scores each frame for change with plain numpy (no model call), and switches to a fast burst when something happens. timelapse/frame_audit.py applies the same scores to an existing frame sequence to tell an acquisition gap, an illumination change or a stage bump from real specimen change. Design, safety model and status: docs/adaptive_timelapse.md.

python -m timelapse.scheduler --backend mock --slow 2 --burst 0.5 --burst-duration 5 --max-runtime 30
python -m timelapse.scheduler --backend mock --model-trigger ...   # Claude judges each trigger (needs API credentials)
python -m timelapse.frame_audit FRAME_DIR --timestamps times.csv --around 25h --window 1h

Tests

pip install -e ".[test]"
python -m pytest -q tests

Mock backend only - nothing touches hardware. CI runs the same on every push.

Install

This is a normal installable package (confocal-mcp) with a console entry point. Dependencies are split into groups so a machine only pulls what it needs:

Group	Pulls in	For
(core)	mcp, Pillow, PyYAML	the MCP server against the mock stage
camera	harvesters, genicam, opencv-python	real Baumer camera capture
sdk	pywin32	real Ti2 stage control (Windows)
harness	anthropic, python-dotenv	the standalone Claude loops in harness/
all	everything above	a full workstation

The real hardware backends are imported lazily, so a core-only install runs fine anywhere (CI, a laptop) - it just can't touch hardware.

From a source checkout

python -m venv .venv
.venv\Scripts\pip install -e ".[camera,sdk]"     # real hardware
.venv\Scripts\pip install -r requirements.txt    # == -e ".[all]"

As a standalone tool (isolated, no repo checkout)

uv tool install "git+https://github.com/BioNanomics/microscope-agent[camera,sdk]"
# or: pipx install "git+https://github.com/BioNanomics/microscope-agent[camera,sdk]"

Real stage backend (one extra step)

acquisition/backends/nis_sdk.py imports NkTi2Ax, the Nikon Ti2 SDK's generated Python bindings - machine-generated (via pywin32's gencache/makepy against the installed SDK), not pip-installable, and not in this repo. If it isn't already in site-packages/, let it regenerate against an installed Ti2 SDK, or copy it from another working environment on the same machine. Not needed for backend="mock".

Run

confocal-mcp                       # installed console script
python -m mcp_server.server_loop   # equivalent, from a source checkout

Point an MCP client at it - Claude Desktop's claude_desktop_config.json, or a project-level .mcp.json for Claude Code:

{ "mcpServers": { "confocal": { "command": "confocal-mcp" } } }

Data (captures, move/frame history logs, saved positions) is written under the current working directory - so launch the server from a stable location, or set CONFOCAL_MCP_DATA_DIR to pin it explicitly:

{ "mcpServers": { "confocal": {
    "command": "confocal-mcp",
    "env": { "CONFOCAL_MCP_DATA_DIR": "D:\\path\\to\\your\\data" }
} } }

Setting it is worth doing for any MCP client, because the client picks the working directory and the caller has no say in it - Claude Desktop on Windows launches servers in C:\WINDOWS\system32. When the working directory is unusable like that, the server falls back to ~/.confocal-mcp and says so on stderr rather than failing at the first capture.

Harness loop (harness/agent.py)

A second, independent way to drive the same tools - a small interactive CLI that calls Claude (Anthropic API) directly with tool use, importing mcp_server/loop_tools.py's functions in-process rather than going through server_loop.py's MCP/stdio protocol. server_loop.py is unaffected either way - use whichever fits: MCP for Claude Desktop/Code, this loop for a standalone script.

Requires ANTHROPIC_API_KEY set - either in a .env file at the repo root (cp .env.example .env, fill in your real key; .env is gitignored and loaded automatically by the harnesses and the model trigger), as a regular environment variable, or via ant auth login. Real hardware (backend="sdk" on move or get_image) always pauses for a live "y/N" approval at the terminal before executing, regardless of what the model requests - see harness/agent.py's header comment for why. Old captured images are pruned from the model's context after a couple of turns (harness/context.py) so a long session doesn't keep resending every frame it has ever captured.

python -m harness.agent

MCP client harness (harness/mcp_agent.py)

A third way to drive the same tools - the real-MCP-protocol counterpart to harness/agent.py. Instead of importing loop_tools.py in-process, this one spawns server_loop.py as a separate subprocess and talks to it exactly the way Claude Desktop/Code do: real MCP over stdio, using the current MCP SDK (mcp==2.0.0, protocol version 2026-07-28). Same .env/ANTHROPIC_API_KEY setup, same live "y/N" real-hardware approval gate, same image pruning via harness/context.py - just reached through an actual client/server boundary instead of a direct function call.

python -m harness.mcp_agent

See docs/mcp_harness.md for the full write-up: how the current MCP protocol differs from what most tutorials show, why confirm is stripped from every tool schema before Claude ever sees it, the MCP↔ Anthropic content-block conversion, and a real stdout/JSON-RPC bug this work found and fixed in nis_mock.py.

Hardware

• Stage/focus: Nikon Ti2-E via the Ti2 ActiveX SDK (nis_sdk.py) - real hardware, no NIS-Elements process required.
• Camera: Baumer VCXU-23C via GenICam/GenTL (baumer_genicam.py) - a separate industrial camera, not the confocal N-SPARC detector. Only one application can hold it open at a time - close Baumer Camera Explorer (or any other GenICam consumer) before running this.

Versioning

Versions follow SemVer; see CHANGELOG.md. The current version is importable as mcp_server.__version__ (from installed package metadata).

License

MIT - see LICENSE.