Physarum imaging and estop - #2
Merged
Merged
Conversation
get_optical_configuration() never recorded the illumination state. It walked OPTICAL_CONFIG_PROPERTIES, a hardcoded list that omitted iDIA_LAMP_Switch and iDIA_LAMP_Pos, so a saved configuration did not say whether the transmitted lamp was on - the one setting that decides whether a camera on the camera port sees anything at all. The iDLED* names the list did carry are ignored by this microscope, whose D-LEDI is not driven through the Ti2 body, so every write to them vanished without an error. Both methods now use the SDK's own DataGet, which fills an INikonTi2AxData with all ~90 properties in one call, keyed by the writable i* names. The hardcoded list is deleted. Pass None as DataGet's first argument - it is declared [in,out] and win32com returns the filled object; the NikonTi2AxData coclass is not registered, so constructing one is a dead end. apply_optical_configuration() gains include_motion=False. The snapshot is now the full device set rather than a curated subset, so without that guard restoring a lamp setting would also drive the stage. Add acquisition/calibration/: - ti2_inventory.py lists every device with its range, unit and Control value, discovering field names from the COM type library and grouping by Control rather than any list written down here. Control is the practical guide to what can be driven: -1 devices (D-LEDI, DIA/EPI/AUX shutters, Intensilight, TIRF, LAPP) reliably ignore writes, >= 0 usually accepts, with measured exceptions (iDIC_PRISM, iTURRET2SHUTTER). Enabled is not a fitted-hardware flag despite the name - it reads True for all 88 devices. - ti2_config.py gives save/show/diff/apply from the command line. apply requires --confirm, skips motion devices unless --include-motion, and polls the read-back instead of reading once, because several devices report the old value for up to a second after a write that did take. Also fixes acquisition/paths.py: runtime data no longer falls back to an unwritable working directory. MCP clients choose that directory and Claude Desktop on Windows uses C:\WINDOWS\system32, so with CONFOCAL_MCP_DATA_DIR unset the first get_image() failed with an access denied error that read as a camera fault.
Decoding the VCXU-23C's BayerRG8 stream with COLOR_BayerRG2RGB inverted R/B; BayerBG2RGB matches the camera's own debayer. get_settings() now reports BalanceWhiteAuto, which persists on the camera across processes.
acquisition/estop.py holds the flag and CLI (engage/status/release). NISSdk.XY_Move, Z_Move and nudge_pfs_offset check it before moving, so no caller can move an axis while it is engaged.
The MCP tool can engage or report the stop but never release it. The server launches the panel on startup (CONFOCAL_NO_ESTOP_PANEL disables).
Walks long moves in 4 mm hops under nis_sdk's 5 mm step limit and asks before moves over 2 mm. XY only; Z stays on the focus knob.
Schedules frames against a fixed start time so capture latency does not accumulate, keeps the camera open for the whole run, and logs per-frame exposure and focus stats to frames.csv.
The e-stop note was inserted mid-sentence into the existing TODO, leaving a truncated copy of its first three lines above the full one.
The docs, original CHM/PDF files and sample scripts come from Nikon's installers and are licensed with SDK access, so they stay local only.
mosaic.json now records mosaic_scale so the scale bar is true on the resized stitched image; older runs fall back to the 0.5 default.
…ounds Per stitched round, segments plasmodium on the agar block and reports area in distance bands from a hand-located oat plus the area-weighted mean distance, so movement toward food is a number, not an impression.
Test A (Z) sends a halt write-back 0.2 s into a 1 mm downward move; Test B (XY) engages the flag during a multi-hop move_xy move. Results on the Ti2-E, 2026-09-25, at 4X: - Z: iZPOSITION reads stayed at the start value for the whole move. The halt read that stale value; its write queued behind the move, which completed all 1000 um, then drove Z back to start. A halt write-back never stops Z - it adds a second, reversed move. - XY: the flag was engaged mid-hop; the next hop ran to its end and the one after was refused. Stopped 0.54 s after the engage, having travelled 12 of 20 mm. Worst case is one hop (HOP_UM = 4 mm).
estop.engage() also wrote the current position back as the target, to freeze an axis mid-move. estop_inflight_test showed it cannot: position reads are cached for the whole move (XY and Z), so the halt commands the START position, and its write queues behind the move. A 1 mm Z move ran to its end and the halt then drove Z all the way back - a second move issued by the stop itself. engage() now only sets the flag. A move in flight runs to its end and the next is refused; move_xy's hops bound the XY overshoot to one hop (measured 0.54 s, <= 4 mm). halt() and the CLI's --no-halt are gone. The STOP panel and the MCP estop tool both go through engage().
…default gated Lets the capture-analyze-decide loop run off the microscope PC. CONFOCAL_MOCK_FRAME_PATH chooses the PNG the mock serves. Both harnesses skip the hardware gate for backend='mock'.
- change_detector.ChangeDetector: model-free per-frame score against a rolling median baseline (pixel diff vs noise, Otsu foreground area delta, validated phase-correlation XY shift). - frame_audit: CLI to audit an existing frame sequence for acquisition gaps, intensity jumps, stage shifts vs. real specimen change - for the 'filming error or biology?' question around the 25h event. - tests/: synthetic-frame tests for both, plus get_image(backend='mock'). - numpy added to core deps; 'test' extra with pytest.
AdaptiveTimelapse drives get_image() on a slow interval, switches to burst mode when ChangeDetector fires, extends on further change, and returns to slow. Stage/sample shifts are logged and never start a burst. Hard caps on captures and runtime. on_trigger hook is the seam for a later model consult. backend='sdk' needs one up-front approval of the whole plan at the CLI; confirm=True is supplied by the scheduler after that, never by a model. Verified live against the mock: swapping the served frame mid-run started a burst within one slow interval, then returned to slow. CI: pytest on 3.11/3.13 with a core+test install, plus an import check.
…ed-interval recorder in the proposal
CI on the pull request failed at pip install on the 3.11 job.
…, every tool over stdio
Spawns python -m mcp_server.server_loop (with the e-stop panel
suppressed for headless CI), lists tools and asserts the set is exactly
{get_image, get_pos, move, get_move_history, estop}, then drives the
mock stage and mock capture and confirms the sdk paths are refused
without confirm - all over real MCP stdio, the path Claude Desktop uses.
Pins the safety property from harness/agent.py and harness/mcp_agent.py: the model never sees 'confirm' in a tool schema; get_image/move on backend='sdk' stop at the gate, a decline returns an error result and executes nothing, an approval injects confirm=True; backend='mock' never prompts and cannot be promoted to real hardware by a smuggled flag. The gate prompt and the tools are stubbed, so no model and no hardware. CI now installs the 'harness' extra so the loops import on the runner.
…ist estop in README harness/agent.py's 'four tools' line is left as is: that in-process harness really does expose only four (it has no estop entry).
… hook
At each burst start/extend the hook shows Claude the frame before, the
frame that fired the detector, and the detector's numbers, and asks for
{decision: extend|ignore, reason}. Extend lengthens the burst, ignore
ends it; anything else is no opinion. Per-run call cap and a minimum
interval between asks. Every failure path returns None so the scheduler
is never worse off than with no hook. The API call is one injectable
function; tests use a fake model. Enabled with --model-trigger.
Scheduler capture events now carry previous_image.
The model call no longer stalls the burst loop. Replies are applied at the next step; extend lengthens the running burst, ignore ends it, a reply after the burst has ended is logged as consult_late. Extensions are monotonic so a detector re-trigger never shortens a window the model lengthened. Verified live on the mock: 0.1 s burst spacing held through a 9 s model call.
The panel only appeared when the MCP server started. A mosaic, timelapse or move_xy run started from a terminal - how the 2026-09-21 run was started - had no STOP button on screen. NISSdk's first stage connection in a process now calls estop.launch_panel(), so the button is up before anything can move, however the run was started. The MCP server uses the same launcher. The panel holds a named mutex for its life; launch_panel() skips spawning when it exists, and a second copy started any other way exits without drawing. CONFOCAL_NO_ESTOP_PANEL=1 still turns it off.
… 3x3 mosaics Setup mode switches to 4x, restores the camera's linear neutral colour state (Camera Explorer had left Gamma 1.45, an adapted colour matrix and R/B gains of 5.9/4.4, saturating every frame), sets exposure, runs a capped focus sweep and centres the worm. The run stitches a 3x3 block each round, finds the worm by size and proximity, and re-centres past a deadband, clamped to a radius around the dish centre.
Segments worms in the TD channel (eggs rejected by shape), measures body length along the skeleton for a rough life stage, and calls the head as the end with more neuronal fluorescence. Defaults to RFP because on the AX it is read out with TD, while a sequentially scanned GFP shows a crawling worm somewhere else.
Each time point is surveyed separately; a Z-stack is collapsed to a max projection for fluorescence and the sharpest plane for TD. Writes one annotated image per time point, worms.csv with a time column and counts.csv with worms and stages per time point.
… any Small larvae are too faint for 4x brightfield but show as a chain of RFP signal, so glowing chains not claimed by a brightfield worm become worms too (orange in the overlay). Brightfield objects with no neuron signal are dropped - every AML18 worm glows. Adds --out, and fails loudly when an overlay cannot be written.
Joins each frame's worms to the nearest similar-length worm of the previous frame, reports distance and speed per track, checks whether the neuron signal fades over the run, and renders the outlined frames as an MP4.
'\c' is an invalid escape; Python 3.12 warns about it and a later release will make it a SyntaxError. The name's value is unchanged.
A JOBS Python task serves /status, /move (relative, capped at 1 mm) and /objective (4x/10x only) on 127.0.0.1:8765 for 10 minutes. NIS macro functions are called through ctypes on g5_regprocs.dll, the way NIS's own limpy calls WaitText; signatures are from the NIS 6.20 macro reference. Nothing moves while the e-stop file exists and Z is never moved. Verified so far: the task binds the port inside nis_ar.exe and answers from outside. The /status, /move and /objective calls have not yet been run on the scope.
Promoted from the one-off script that scored the Physarum Short. The arrangement is unchanged and stretches to any length; --movie sizes the track to a movie and muxes it on (video copied, AAC audio) with the ffmpeg bundled by imageio-ffmpeg. At 39 s the output is byte-identical to the original track.
Collaborator
Author
|
C.Elegans Short: https://www.youtube.com/shorts/cvniTfc1r-I |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.