Skip to content

setup.md

Latest commit

 

History

History
188 lines (140 loc) · 6.46 KB

setup.md

File metadata and controls

188 lines (140 loc) · 6.46 KB
Raw

End-to-end setup

This guide installs the laptop services, builds the original face assets, pairs the custom firmware with the server, and flashes an M5Stack StackChan K151 while preserving the Wi-Fi credentials already stored in its factory NVS partition.

1. Requirements

  • Apple Silicon Mac running macOS
  • M5Stack StackChan K151 with CoreS3 and a data-capable USB-C cable
  • Stack-chan and the Mac on the same trusted Wi-Fi network
  • Pixi
  • Homebrew and whisper-cpp
  • Codex CLI authenticated with the account that will run Eve

Install the two host prerequisites that are not distributed by this project:

curl -fsSL https://pixi.sh/install.sh | sh
brew install whisper-cpp
codex login

The active pipeline does not use Docker or Ollama.

2. Install the project and local speech models

From the repository root:

pixi install
pixi run intelligence-install
pixi run download-models

download-models installs the three pinned whisper.cpp files into the ignored artifacts/models/ directory and verifies their SHA-256 checksums. Supertonic, FFmpeg, Python, Node.js, PlatformIO, and the test tools come from the Pixi lock.

Confirm that Homebrew exposed both whisper.cpp programs:

command -v whisper-cli
command -v whisper-server

If Homebrew uses a nonstandard prefix, set STACKCHAN_WHISPER_CLI and STACKCHAN_WHISPER_SERVER in server/.env to their absolute paths.

3. Create private local configuration

cp server/.env.example server/.env
cp firmware/include/LocalConfig.example.hpp firmware/include/LocalConfig.hpp
pixi run provision-device

provision-device creates one random pairing token and writes it only to:

  • secrets/device-token.txt
  • firmware/include/DeviceSecret.hpp
  • server/.env

All three destinations are ignored by Git. The command never prints the token.

Find the Mac's Bonjour name:

scutil --get LocalHostName

Edit firmware/include/LocalConfig.hpp so STACKCHAN_SERVER_HOST is that value with .local appended. Leave port 8765 and path /v1/device unchanged unless the server configuration is changed too. Allow incoming connections for the Python process if the macOS firewall prompts.

4. Build and flash without erasing saved Wi-Fi

Connect Stack-chan over USB, then run:

pixi run firmware-build
pixi run firmware-upload

The firmware reads ssid and password from the existing read-only wifi NVS namespace. Its own boot metadata uses a separate stackchan-meta namespace. The normal PlatformIO upload updates program images and leaves the NVS data partition intact.

Do not run an erase-flash command, change the partition layout, or manually write over the NVS partition if the saved Wi-Fi must be preserved. A full-flash backup, if you choose to keep one, belongs outside this repository because it can contain network credentials.

5. Start Stack-chan

pixi run start

This one command starts Eve, the resident Whisper and Supertonic services, SQLite memory, and the authenticated robot WebSocket server. It prints a short readiness summary and waits for Stack-chan to reconnect. Press Ctrl-C to stop every service it started; detailed logs are under artifacts/logs/.

Raw microphone audio, speech models, TTS, and SQLite memory stay on the Mac. Final transcripts, selected memory/context, and tool schemas are sent to the configured Eve model.

Optional attentive listening with MaAI

MaAI can predict well-timed backchannels in English/Japanese and three kinds of Japanese nod. It runs in a separate Pixi environment and subprocess, consumes only the existing 16 kHz AEC-clean microphone and physical-render streams, and drops stale frames instead of delaying STT, TTS, or interruption handling.

pixi install -e maai

Then set STACKCHAN_MAAI_ENABLED=true in server/.env and restart. Keep STACKCHAN_MAAI_SHADOW_MODE=true while checking optional.maai in /health and maai_inference entries in the device results endpoint. After validating the room and microphone, set shadow mode to false to enable sparse two-step head acknowledgements. MaAI never creates an LLM turn or speaks over the user.

Benchmark the isolated bridge with:

pixi run benchmark-maai

The MaAI code and the selected vap_bc_jp, vap_bc_en, and vap_nod_jp checkpoints are MIT-licensed upstream dependencies downloaded into ignored local caches; they are not redistributed by this repository.

6. Validate before normal use

Run the deterministic suite first:

pixi run check

With the physical robot connected to the running services, the focused current acceptance checks are:

pixi run hil-recent-regressions
pixi run hil-no-false-barge
pixi run hil-sensor
pixi run hil-routine-music
pixi run hil-music-styles
pixi run hil-camera
pixi run hil-daily-routines
pixi run hil-face-requests

The HIL tasks can move the servos, illuminate body LEDs, play audio, and capture one locally stored camera still. Keep the robot on a clear surface and do not force a powered servo by hand. Vertical motion is constrained to the hardware-safe 5-85 degree range.

Camera captures require an explicit photo command, use a visible white-light cue, and are stored only in ignored artifacts/captures/. On macOS, the server automatically compiles the included local Vision helper when scene analysis is needed; no camera image is sent to Eve or a cloud vision model.

Repeatable far-field voice interruption is still experimental and has been deferred. Do not use hil-voice-benchmark as an unattended pass/fail gate yet.

Troubleshooting

  • Face says Reconnecting: start both laptop services, verify the Bonjour host in LocalConfig.hpp, and confirm both devices are on the same network.
  • Connected but authentication fails: rerun pixi run provision-device, rebuild/upload the firmware, and restart the server so both sides use the same locally stored token.
  • Health reports Whisper missing: install whisper-cpp, verify the two binaries above, then run pixi run download-models again.
  • Idle but no speech: inspect /health, then the ignored logs under artifacts/logs/; verify Supertonic and all three Whisper lanes are ready.
  • Wi-Fi no longer connects after an erase: restore credentials outside this project. The custom firmware intentionally has no API that reveals them.

See architecture.md, protocol.md, and completion-audit.md for design and evidence details.