diff --git a/README.md b/README.md index 6fcddf0..4c04830 100644 --- a/README.md +++ b/README.md @@ -39,6 +39,14 @@ Want the turnkey, print-and-go version? See [`docs/BuildKit.md`](docs/BuildKit.m Need the fastest “we’re alive” path? The first‑play ritual lives in [`docs/FirstSession.md`](docs/FirstSession.md). It’s a 10‑minute, no‑excuses script: connect hardware, flash firmware, open serial, run the bridge, fire up the visualizer, and paste a known‑good gesture packet to prove the loop. +## 90‑minute onboarding bootcamp + +Want a structured ramp for new collaborators? [`docs/Onboarding90Min.md`](docs/Onboarding90Min.md) is a one‑session syllabus: get the pipeline working, narrate sensors, tune one gesture knob, and leave able to teach the system. + +## Gesture vocabulary (read‑aloud heuristics) + +Need the “say it out loud” version of the gesture engine? [`docs/GestureVocabulary.md`](docs/GestureVocabulary.md) ties the human feel of pluck/bow/scrape/harmonic/mute/tremolo/vibrato to the exact firmware knobs. + ## Runtime note-set auditions (Serial preset browser) When you hear someone shout “try it in Hirajōshi!”, you no longer have to recompile. diff --git a/docs/GestureVocabulary.md b/docs/GestureVocabulary.md new file mode 100644 index 0000000..aaf3475 --- /dev/null +++ b/docs/GestureVocabulary.md @@ -0,0 +1,133 @@ +# Gesture Vocabulary (Narratable Heuristics) + +_This is the “read it out loud” guide to the gesture engine. It explains **why** each gesture exists, **how** we detect it, and **which knobs** to tune when reality gets noisy. Pair this with `firmware/src/gesture_engine.cpp` and the parameter table in `firmware/include/gesture_engine.h`._ + +## Big idea: gestures are stories, not equations + +We don’t look for “perfect signals.” We look for **behavior** that players can repeat and instructors can narrate. Every gesture is a small, human‑readable rule with a few thresholds you can tweak while the room listens. + +--- + +## Gesture quick map (what the engine calls it) + +| Gesture | How it feels | What the engine looks for | +| --- | --- | --- | +| **Pluck** | A single, intentional attack | Contact crosses `on_thresh` after enough time since the last onset | +| **Scrape** | Rapid, grainy micro‑attacks | Two onsets closer than `scrape_window_us` | +| **Bow** | Sustained touch / continuous motion | Continuous contact, no special harmonic/tremolo trigger | +| **Harmonic** | Light, steady touch with “glass” tone | Peak stays within `harmonic_peak_min`..`max` **and** stays still for `harmonic_hold_us` | +| **Muted** | Short, damped touch | Release within `mute_window_us` **or** drop below `mute_release_thresh` | +| **Tremolo** | Shallow, fast oscillation | Wobble count hits `wobble_goal` with low depth | +| **Vibrato** | Deeper oscillation | Wobble count hits `wobble_goal` with depth ≥ `vibrato_depth_min` | + +--- + +## The narratable rules (human language first) + +### 1) Pluck + +**Say this:** +“A pluck is a clear crossing into contact. If it happens too soon after the last one, we ignore it. If it’s close enough to a prior onset, we call it a scrape instead.” + +**Knobs that matter:** +`on_thresh`, `off_thresh`, `min_retrigger_us`, `scrape_window_us` + +--- + +### 2) Scrape + +**Say this:** +“A scrape is just a **rapid series of onsets**. The time between them is shorter than a pluck.” + +**Knobs that matter:** +`scrape_window_us` (shorter = stricter, longer = more scrapes) + +--- + +### 3) Bow + +**Say this:** +“Bow is the default: it’s what we call continuous contact when nothing else is true. It’s sustained energy with no special wobble or harmonic signature.” + +**Knobs that matter:** +Mostly `on_thresh` and `off_thresh`, plus smoothing in your sensor front‑end. + +--- + +### 4) Harmonic + +**Say this:** +“A harmonic is a **light, steady touch**. You hit a gentle peak, then hold it still long enough to earn the glassy tone.” + +**Knobs that matter:** +`harmonic_peak_min`, `harmonic_peak_max`, `harmonic_hold_us`, `harmonic_variation_eps` + +**Calibration ritual:** + +1. Lightly graze the sensor. +2. Nudge `harmonic_peak_min`/`max` until harmonics trigger without full plucks. +3. Tighten `harmonic_variation_eps` until the system demands stillness. + +--- + +### 5) Muted + +**Say this:** +“A mute is a **short, damped touch**. If the contact ends quickly, or the signal drops below the mute floor, we call it muted.” + +**Knobs that matter:** +`mute_peak_thresh`, `mute_window_us`, `mute_release_thresh` + +**Teaching cue:** +Ask the class to “tap and vanish” — quick contact then immediate release. + +--- + +### 6) Tremolo + +**Say this:** +“Tremolo is a **fast, shallow wobble**. The engine counts sign flips and calls it once it sees enough.” + +**Knobs that matter:** +`tremolo_min_delta`, `tremolo_max_period_us`, `tremolo_grace_us`, `wobble_goal` + +--- + +### 7) Vibrato + +**Say this:** +“Vibrato is tremolo with more depth. Same wobble count, bigger swing.” + +**Knobs that matter:** +`vibrato_depth_min` (raise it to demand bigger swings) + +--- + +## Threshold cheat‑sheet (firmware knobs) + +These live in `firmware/include/gesture_engine.h`. Keep these names in your teaching script so students can go from “feel” to “code” without translation. + +- **Contact gates:** `on_thresh`, `off_thresh` +- **Time guards:** `min_retrigger_us`, `scrape_window_us`, `harmonic_hold_us`, `mute_window_us`, `tremolo_grace_us` +- **Harmonic band:** `harmonic_peak_min`, `harmonic_peak_max`, `harmonic_variation_eps` +- **Mute window:** `mute_peak_thresh`, `mute_release_thresh` +- **Wobble rules:** `tremolo_min_delta`, `tremolo_max_period_us`, `wobble_goal`, `vibrato_depth_min` + +--- + +## In‑class debugging prompts (keep it human) + +- “Is the sensor normalized to 0..1?” (Check your sensor front‑end.) +- “Do we see the peak hit the harmonic band?” (Plot the envelope.) +- “Is the wobble count too strict?” (Lower `wobble_goal` or `tremolo_min_delta`.) +- “Are we double‑triggering?” (Raise `min_retrigger_us`.) + +--- + +## Next expansion ideas (for brave collaborators) + +- **Glissando**: detect steady contact with a monotonic rise/fall in value. +- **Ghost note**: sub‑threshold taps that light the visualizer but don’t send MIDI. +- **Scrape texture index**: count scrape grains per second and map it to CC. + +If you prototype a new gesture, describe it here first. Code comes second. diff --git a/docs/Onboarding90Min.md b/docs/Onboarding90Min.md new file mode 100644 index 0000000..1267e12 --- /dev/null +++ b/docs/Onboarding90Min.md @@ -0,0 +1,141 @@ +# 90‑Minute Onboarding Bootcamp (StringField Edition) + +_A brisk, teachable ramp for newcomers: half studio notebook, half workshop script. Use it solo or read it aloud while a room builds the instrument together._ + +## The promise + +In 90 minutes you’ll: + +- **Play** a gesture (pluck/bow/scrape) and see it become motion on‑screen. +- **Understand** how sensors become a normalized signal (0..1). +- **Name** the gesture heuristics in human language, not math‑speak. +- **Adjust** at least one calibration knob on purpose. + +If you finish this, you can _teach_ StringField, not just run it. + +--- + +## 0. Pre‑flight (5 minutes) + +**You need:** + +- A StringField sensor stack + Teensy/ESP32 (or whatever hardware you’ve got). +- USB cable. +- PlatformIO + Processing installed. + +**Open these files now:** + +- `docs/FirstSession.md` (fast path ritual). +- `firmware/src/gesture_engine.cpp` (gesture logic). +- `firmware/include/gesture_engine.h` (tunable thresholds). +- `software/p5js/README.md` or `software/processing/StringFieldViz.pde` (your visualizer). + +--- + +## 1. The 10‑minute “it works” ritual (10 minutes) + +Follow `docs/FirstSession.md` end‑to‑end. Don’t skip steps. The pipeline is the instrument: + +**hardware → firmware → serial → bridge → visualizer** + +If you see the `PLUCK` state in the visualizer ticker, you are live. + +--- + +## 2. Sensor signal = normalized story (15 minutes) + +Open the sensor implementation you’re using: + +- `firmware/src/optical_sensor.cpp` +- `firmware/src/capacitive_sensor.cpp` +- `firmware/src/makey_sensor.cpp` +- `firmware/src/time_of_flight_sensor.cpp` +- `firmware/src/piezo_sensor.cpp` +- `firmware/src/pir_sensor.cpp` +- `firmware/src/electret_mic_sensor.cpp` +- `firmware/src/i2s_mic_sensor.cpp` + +**Narrate this out loud while you read:** + +1. **What raw signal comes in?** (ADC, digital gate, PCM samples) +2. **How do we normalize it?** (map into 0..1) +3. **How do we smooth it?** (low‑pass, envelope, debounce) +4. **What breaks it?** (grounding, ambient light, sensor saturation) + +Bonus punk‑rock move: open `docs/Sensors/README.md` and read the failure modes like a checklist. Tech is a performance—own the glitches. + +--- + +## 3. Gesture vocabulary anatomy (15 minutes) + +Open `docs/GestureVocabulary.md`. This is your teaching script. + +Key files to cross‑reference: + +- `firmware/include/gesture_engine.h` (the tuning knobs) +- `firmware/src/gesture_engine.cpp` (the rules) + +**Do one live tweak** (even if it’s small): + +- Raise `harmonic_hold_us` to make harmonics harder. +- Lower `tremolo_min_delta` to make tremolo easier. +- Tighten `scrape_window_us` to cut accidental scrapes. + +Reflash, replay, and narrate what changed. That’s the whole pedagogy loop. + +--- + +## 4. Visualize the signal (15 minutes) + +Pick one: + +- **p5.js debugger:** `software/p5js/sketch.js` + (Teach “data in → motion out” with big labels and velocity bars.) +- **Processing viz:** `software/processing/StringFieldViz.pde` + (Project it; the room needs to see the signal breathe.) + +**Prompt:** “What does a ‘bow’ _look_ like on screen?” +Let the room answer before you explain it. + +--- + +## 5. Hardware‑aware tuning (15 minutes) + +Choose the sensor page that matches your build and follow the calibration ritual: + +- `docs/Sensors/OpticalReflective.md` +- `docs/Sensors/CapacitiveSingleWire.md` +- `docs/Sensors/MakeyTouch.md` +- `docs/Sensors/TimeOfFlight.md` +- `docs/Sensors/Piezo.md` +- `docs/Sensors/PIR.md` +- `docs/Sensors/ElectretMic.md` +- `docs/Sensors/I2SMic.md` + +**Goal:** adjust one knob in firmware that maps to a physical sensation. +Example: “Raise `on_thresh` until the bow only activates when you _mean_ it.” + +--- + +## 6. Exit ticket (15 minutes) + +By the end, make sure you can answer (or teach) these questions: + +- What does 0..1 mean for your current sensor? +- What separates a pluck from a scrape in code? +- How does the gesture engine avoid false retriggers? +- Which knob would you tune first in a noisy room? + +If you can narrate this without staring at the code, you win. + +--- + +## Optional homework (30–60 minutes) + +Pick one: + +- Run the gesture tests: `pio test -d firmware -e native` +- Add a new sensor note page in `docs/Sensors/` +- Prototype a new “gesture” (e.g., glissando) by sketching rules in `docs/GestureVocabulary.md` + +If you write it down, it’s teachable. If it’s teachable, it’s real. diff --git a/docs/SensingSurvey.md b/docs/SensingSurvey.md index e4f4bb7..1d23b26 100644 --- a/docs/SensingSurvey.md +++ b/docs/SensingSurvey.md @@ -16,6 +16,8 @@ Each of these is a short, hands‑on guide with wiring notes, RC recipes, calibration rituals, and “what the gestures should feel like.” - Start here for the index: [Sensor Field Notes](Sensors/README.md) +- [Optical Reflective IR](Sensors/OpticalReflective.md) +- [Capacitive Single‑Wire](Sensors/CapacitiveSingleWire.md) - [MaKey‑style Touch‑to‑Ground](Sensors/MakeyTouch.md) - [Time‑of‑Flight Proximity](Sensors/TimeOfFlight.md) - [Piezo / Contact Mic](Sensors/Piezo.md) diff --git a/docs/Sensors/CapacitiveSingleWire.md b/docs/Sensors/CapacitiveSingleWire.md new file mode 100644 index 0000000..f43cab4 --- /dev/null +++ b/docs/Sensors/CapacitiveSingleWire.md @@ -0,0 +1,44 @@ +# Capacitive Single‑Wire (Field Notes) + +_Goal: make an invisible string using a single wire + grounding ritual. This modality is sensitive and expressive, but it **demands** a good ground story._ + +## What it feels like + +Capacitive single‑wire sensing is **ghost‑string** mode: the player doesn’t have to touch a pad; just approaching the wire changes the capacitance. It’s soft, quiet, and beautifully expressive—but it’s also moody when humidity shifts or grounding is sloppy. + +--- + +## Wiring sketch (prototype‑level) + +- **Sensor wire:** a long lead or foil strip to `A1` +- **Ground reference:** wearable bracelet or a shared foil pad to GND +- **No fancy IC required:** this is the simplest RC‑rise trick (read the pin, discharge it, charge it, read it again) + +**Expected signal range:** analog 0..1023 (ADC). +**Firmware file:** `firmware/src/capacitive_sensor.cpp` + +--- + +## Calibration ritual (4 minutes) + +1. **Check the baseline:** no one touches the wire. You should see a stable, low signal. +2. **Hover and touch:** approach the wire, then touch. The delta should grow. +3. **Adjust sensitivity:** tweak `sensitivity_scale_` (in `firmware/src/capacitive_sensor.cpp`) until the envelope hits 0.7–0.9 on intentional touches but stays near 0 at rest. + +--- + +## Narratable failure modes + +- **Always maxed out:** missing or bad ground. The body is not part of the RC circuit, so the pad floats high. +- **Flatline:** wire broken or pin mis‑mapped; verify `A1`. +- **Noisy crawl:** humidity swings. Lower the baseline follower rate or re‑run calibration mid‑session. + +--- + +## Teach the normalization + +Say this: + +> “We’re not measuring distance directly. We’re measuring how long a tiny RC circuit takes to charge. Then we map that to 0..1 so the gesture engine doesn’t care what the sensor is.” + +If students get that, they can debug it. diff --git a/docs/Sensors/OpticalReflective.md b/docs/Sensors/OpticalReflective.md new file mode 100644 index 0000000..a0dc98d --- /dev/null +++ b/docs/Sensors/OpticalReflective.md @@ -0,0 +1,45 @@ +# Optical Reflective IR (Field Notes) + +_Goal: a fast, cheap “string in air” that’s readable in class. This page is meant to be read out loud while you wire._ + +## What it feels like + +Reflective IR is **bow‑friendly** and **scrape‑friendly**. It catches continuous motion better than micro‑taps, and it’s sensitive to ambient light, surface albedo, and room layout. Use it when you want the “string” to be visible and performative. + +--- + +## Wiring sketch (prototype‑level) + +- **Emitter (IR LED):** 5V → resistor → IR LED → GND + (pick a resistor that keeps the LED cool: start with 100–220Ω) +- **Receiver (phototransistor):** collector → 5V, emitter → resistor → GND + tap the emitter (or resistor midpoint) into `A0` + +**Expected signal range:** analog 0..1023 (ADC). +**Firmware file:** `firmware/src/optical_sensor.cpp` + +--- + +## Calibration ritual (3 minutes) + +1. **Dark baseline:** cover the sensor, watch the serial plotter. That’s your “quiet” floor. +2. **Hand at distance:** hover at your intended playing distance. You should see the envelope rise. +3. **Thresholds:** raise `on_thresh` until light chatter stops, then lower it just enough to catch intentional contact. + +--- + +## Narratable failure modes (aka “what’s wrong” in class) + +- **Always high:** ambient light or a reflective surface is flooding the sensor. Shield it, angle it down, or move away from windows. +- **Always low:** emitter LED wired backwards or dead. Check polarity and resistor. +- **Noisy flicker:** fluorescent lighting or PWM LEDs nearby; increase smoothing or add a light shield. + +--- + +## Teach the normalization + +Explain it like this: + +> “We read a raw voltage, squeeze it into 0..1, subtract a moving ambient baseline, and smooth it so the gesture engine sees _motion energy_ instead of raw flicker.” + +That one sentence turns a blinking LED into a teachable instrument. diff --git a/docs/Sensors/README.md b/docs/Sensors/README.md index 084be5e..38bf330 100644 --- a/docs/Sensors/README.md +++ b/docs/Sensors/README.md @@ -6,6 +6,8 @@ If you want the fastest path: pick a sensor below, wire it, compile with the mat ## The sensor menu +- **Optical reflective IR:** [`OpticalReflective.md`](OpticalReflective.md) +- **Capacitive single‑wire:** [`CapacitiveSingleWire.md`](CapacitiveSingleWire.md) - **MaKey‑style touch‑to‑ground:** [`MakeyTouch.md`](MakeyTouch.md) - **Time‑of‑flight proximity:** [`TimeOfFlight.md`](TimeOfFlight.md) - **Piezo / contact mic:** [`Piezo.md`](Piezo.md) diff --git a/firmware/src/optical_sensor.cpp b/firmware/src/optical_sensor.cpp index 1fba45d..317d15b 100644 --- a/firmware/src/optical_sensor.cpp +++ b/firmware/src/optical_sensor.cpp @@ -11,14 +11,35 @@ class OpticalSensor : public Sensor { } SensorSample read() override { + // Expected signal range: analog 0..1023 from a phototransistor divider. + // If you see 0 or 1023 all the time, check wiring and whether the sensor is saturated. int raw = analogRead(A0); // 0..1023 on Teensy (will be 12-bit on some MCUs) - // Map raw to 0..1 with a crude low-pass to tame flicker; tune in calibration. - static float y = 0.0f; float x = constrain(raw / 1023.0f, 0.0f, 1.0f); - y = 0.9f * y + 0.1f * x; + + // Ambient light baseline: follow slow changes in the room without chasing the hand. + if (x < ambient_floor_) { + ambient_floor_ = 0.95f * ambient_floor_ + 0.05f * x; // drop quickly if the room gets darker + } else { + ambient_floor_ = 0.995f * ambient_floor_ + 0.005f * x; // rise slowly so hands don't become "baseline" + } + + // Normalize: subtract the ambient floor, overdrive slightly for expressive motion. + float normalized = (x - ambient_floor_) * gain_; + normalized = constrain(normalized, 0.0f, 1.0f); + + // Low-pass smoothing to tame flicker without erasing intentional motion. + env_ = 0.9f * env_ + 0.1f * normalized; + + // Common failure modes: sunlight swamping the sensor, reflective surfaces + // causing false positives, or the emitter LED wired backward (flatline). digitalWrite(13, (millis() >> 6) & 1); // slow blink to show life - return {y, micros()}; + return {env_, micros()}; } + + private: + float ambient_floor_ = 0.0f; + float env_ = 0.0f; + const float gain_ = 1.4f; // bump for low-contrast rooms; adjust in class }; Sensor& get_optical_sensor() { @@ -27,4 +48,3 @@ Sensor& get_optical_sensor() { } #endif // SENSOR_OPTICAL -