Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
d8815ca
brought nanocockpit aideck drivers here
cookacola Jun 26, 2026
30db53a
UART receiver on STM side implemented
cookacola Jun 26, 2026
938d662
ancillary commits
cookacola Jun 26, 2026
4b817b0
Implemented PnP + pipeline that takes in four corner coordinates and …
cookacola Jun 27, 2026
f4162df
Successful hardware testing where drone recognizes gate and flies thr…
cookacola Jun 28, 2026
853f5c7
Attempt to use vision to aid state estimation. Unsuccessful when turn…
cookacola Jun 29, 2026
40d45b9
Gradual integration of changes to pathing
cookacola Jul 7, 2026
9f12513
High level TinyMPC - Low level PID - Working Yaw
cookacola Jul 10, 2026
d188f37
Stage 1: gate perception in the cascade controller
cookacola Jul 13, 2026
c40ad49
Stage 2: fly a trajectory through the detected gate
cookacola Jul 13, 2026
f49d937
Stage 3: two-gate surveyed racetrack circuit
cookacola Jul 13, 2026
55cdcab
Add camera-accuracy-vs-mocap verification tooling
cookacola Jul 15, 2026
f14218a
Obstacle detection pre-setup
cookacola Jul 20, 2026
cf2644c
Implementation doc and tooling added
cookacola Jul 20, 2026
cb59d88
STM side of obstacle avoidance
cookacola Jul 21, 2026
28f1960
Simulation obstacle avoidance debugging pipeline
cookacola Jul 21, 2026
56fb7ec
Fake obstacles avoided
cookacola Jul 22, 2026
4f91ca2
Uses estimator velocity from state->velocity as well as gyro
cookacola Jul 22, 2026
b458cd8
3 feet away seems relatively accurate
cookacola Jul 22, 2026
eef307c
Kalman style estimation of points
cookacola Jul 22, 2026
a143f6f
Fix zeroing of cylinder
cookacola Jul 22, 2026
a97944b
Integration of generated ellipse into obstacle constraints test
cookacola Jul 22, 2026
244bfb2
Faulty run, but run nevertheless
cookacola Jul 22, 2026
86a3743
Make more faithful to demo-2
cookacola Jul 22, 2026
8318fb8
Semaphore fix
cookacola Jul 22, 2026
45a8091
50hz fix
cookacola Jul 22, 2026
cd0b034
50hz fix pt2
cookacola Jul 22, 2026
c75ffd9
Add flow obstacle mapping and TinyMPC obstacle flight test
cookacola Jul 22, 2026
2be3188
Simulation testing crazyflie firmware for optimal camera algo
cookacola Jul 23, 2026
c12edaf
Use AI-deck UART firmware revision
cookacola Jul 23, 2026
084ea99
Harden flow obstacle detection and flight validation
cookacola Jul 23, 2026
1f05adb
Synchronize obstacle flow with capture state
cookacola Jul 24, 2026
a27c38f
Receive versioned feature track packets
cookacola Jul 24, 2026
a5786cd
Drive obstacle perception from feature tracks
cookacola Jul 24, 2026
f43989b
Expand exact-source obstacle position coverage
cookacola Jul 24, 2026
0df290e
Detect race obstacles from forward optical flow
cookacola Jul 24, 2026
019c147
Resolve with remote
cookacola Jul 24, 2026
96c214a
Add continuous confined-course perception replay
cookacola Jul 24, 2026
fd0965d
Add AI-deck peer approach capture runner
cookacola Jul 24, 2026
6be09eb
Fix peer approach capture startup
cookacola Jul 24, 2026
09fd920
Reduce AI-deck capture frame drops
cookacola Jul 24, 2026
a9437c9
Reject stale capture gaps in flow replay
cookacola Jul 24, 2026
ebb364e
Replay low-drop Himax PGM captures
cookacola Jul 24, 2026
91d4bc4
Stabilize obstacle cylinder association
cookacola Jul 24, 2026
d3fb80d
Correct mirrored Himax flow geometry
cookacola Jul 24, 2026
657804f
Pin improved peer capture acceptance
cookacola Jul 24, 2026
61f9043
Freeze flow obstacle on support gate
cookacola Jul 24, 2026
46e1a08
Handle stale telemetry during flow flight
cookacola Jul 24, 2026
6848da8
Relax flow flight abort gates
cookacola Jul 24, 2026
8b40e78
Add ADMM run after flow freeze
cookacola Jul 24, 2026
f2e5bfd
Require distinct flow packets for freeze hold
cookacola Jul 24, 2026
afb348a
Add exact race perception replay flight mode
cookacola Jul 24, 2026
e00c2c0
Integrate gate-aware perception with TinyMPC
cookacola Jul 28, 2026
62f76f1
Consume calibrated STDC danger maps
cookacola Jul 28, 2026
4795c83
Calibrate shared STDC danger threshold
cookacola Jul 29, 2026
59613c7
Merge remote-tracking branch 'origin/vision' into vision
cookacola Jul 29, 2026
edeebdd
Checkpoint image-guided obstacle avoidance
cookacola Jul 30, 2026
10096eb
Revert "Checkpoint image-guided obstacle avoidance"
cookacola Jul 30, 2026
6b25d97
Remove obsolete flow and EKF paths
cookacola Jul 30, 2026
53629dc
Add sequential vision obstacle avoidance
cookacola Aug 2, 2026
6dfdd73
Build controller before flashing
cookacola Aug 2, 2026
22f015e
Update sequential obstacle flight script
cookacola Aug 4, 2026
2321b0f
Add averaged obstacle avoidance trigger
cookacola Aug 4, 2026
3fff849
Add sequential obstacle avoidance controls
cookacola Aug 4, 2026
bda91ae
Prune legacy controller paths
cookacola Aug 4, 2026
dd3ff8f
Remove obsolete controller Python tools
cookacola Aug 4, 2026
516bda6
Integrate generated upstream TinyMPC parameters
cookacola Aug 5, 2026
1ebd553
Refine obstacle avoidance simulation tools
cookacola Aug 5, 2026
c71e4c5
Restore old trajectory methods
cookacola Aug 5, 2026
e33b780
Stabilize constrained TinyMPC planning and reorganize tools
cookacola Aug 5, 2026
7efc3e8
Checkpoint vision-guided half-space avoidance
cookacola Aug 6, 2026
9aefff8
Improve vision-guided trajectory avoidance
cookacola Aug 6, 2026
0091e2c
World -> Local frame transformation
cookacola Aug 6, 2026
bb06dd9
Update TinyMPC direct-control obstacle constraints
cookacola Aug 6, 2026
f102f2a
Add TinyRacer constrained obstacle avoidance
cookacola Aug 7, 2026
481260b
Add high-fidelity TinyMPC maneuver simulation
cookacola Aug 19, 2026
c47840a
Add offline stored LTV maneuver models
cookacola Aug 19, 2026
27c578a
Validate direct-motor acrobatics in CrazySim
cookacola Aug 20, 2026
7a67f34
Rebuild firmware vision simulation with DroNet baseline
cookacola Aug 20, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
Binary file added .DS_Store
Binary file not shown.
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,9 @@ build/
*.tmp
*.o
*:Zone.Identifier
.DS_Store
__pycache__/
*.py[cod]
.cf_cache/
apps/controller_tinympc_eigen/sim_runs/
apps/controller_tinympc_eigen/tools/debug_runs/
4 changes: 2 additions & 2 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
url = https://gitlab.com/libeigen/eigen.git
[submodule "TinyMPC"]
path = TinyMPC
url = https://github.com/moisesmata/TinyMPC
url = https://github.com/TinyMPC/TinyMPC.git
[submodule "crazyflie-firmware"]
path = crazyflie-firmware
url = https://github.com/moisesmata/crazyflie-firmware.git
url = https://github.com/cookacola/crazyflie-firmware.git
375 changes: 375 additions & 0 deletions OBSTACLE_PERCEPTION_HANDOFF.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion TinyMPC
Binary file added apps/.DS_Store
Binary file not shown.
1 change: 1 addition & 0 deletions apps/controller_tinympc_eigen/.gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
bin/*
cf2.*
build/*
.cf_cache/
36 changes: 36 additions & 0 deletions apps/controller_tinympc_eigen/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,37 @@ EXTRA_CFLAGS += -DEIGEN_INITIALIZE_MATRICES_BY_ZERO
EXTRA_CFLAGS += -DEIGEN_NO_MALLOC
EXTRA_CFLAGS += -DNDEBUG
EXTRA_CFLAGS += -Wno-unused-result
# The capture-time state channel uses a dedicated UART1 DMA sender task.
EXTRA_CFLAGS += -DENABLE_UART1_DMA

# Select the compiled stored reference. The default remains the conservative
# circle; the four acrobatic choices enable reference-centered quaternion-error
# TinyMPC and include per-motor feedforward in the trajectory header.
TINYMPC_TRAJECTORY ?= circle
TINYMPC_STORED_LTV ?= 0
ifeq ($(TINYMPC_TRAJECTORY),roll_flip_360)
EXTRA_CFLAGS += -DTINYMPC_TRAJECTORY_ROLL_FLIP_360=1
else ifeq ($(TINYMPC_TRAJECTORY),front_flip_360)
EXTRA_CFLAGS += -DTINYMPC_TRAJECTORY_FRONT_FLIP_360=1
else ifeq ($(TINYMPC_TRAJECTORY),backflip_360)
EXTRA_CFLAGS += -DTINYMPC_TRAJECTORY_BACKFLIP_360=1
else ifeq ($(TINYMPC_TRAJECTORY),barrel_roll_forward_360)
EXTRA_CFLAGS += -DTINYMPC_TRAJECTORY_BARREL_ROLL_FORWARD_360=1
else ifeq ($(TINYMPC_TRAJECTORY),straight)
EXTRA_CFLAGS += -DTINYMPC_TRAJECTORY_STRAIGHT=1
else ifeq ($(TINYMPC_TRAJECTORY),figure8)
EXTRA_CFLAGS += -DTINYMPC_TRAJECTORY_FIGURE8=1
else ifneq ($(TINYMPC_TRAJECTORY),circle)
$(error Unsupported TINYMPC_TRAJECTORY '$(TINYMPC_TRAJECTORY)')
endif
ifeq ($(TINYMPC_STORED_LTV),1)
ifeq ($(TINYMPC_TRAJECTORY),circle)
$(error TINYMPC_STORED_LTV=1 requires an acrobatic TINYMPC_TRAJECTORY)
endif
EXTRA_CFLAGS += -DTINYMPC_USE_STORED_LTV=1
else ifneq ($(TINYMPC_STORED_LTV),0)
$(error TINYMPC_STORED_LTV must be 0 or 1)
endif
#
# We override the default OOT_CONFIG here, we could also name our config
# to oot-config and that would be the default.
Expand All @@ -47,3 +78,8 @@ OOT_CONFIG := $(CURDIR)/app-config
OOT_USES_CXX := 1

include $(CRAZYFLIE_BASE)/tools/make/oot.mk

# The upstream OOT `cload` target only transfers the existing binary. Make
# flashing safe after a source update: build the app first so `make cload`
# cannot silently reinstall a stale cf21bl.bin lacking new parameters/logs.
cload: all
132 changes: 130 additions & 2 deletions apps/controller_tinympc_eigen/README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,50 @@
# TinyMPC Eigen Controller

This is the supported out-of-tree TinyMPC controller on `main`.
This is the supported out-of-tree TinyMPC controller on `main`. It has been further modified to use Nanocockpit's aideck drivers to facilitate better communication with the aideck and aid in vision tasks.

## Build

Run the build from this directory:

Note: In order to use the new drivers, you must set `CONFIG_DECK_AI=n` in your app's app-config/oot-config to dropthe old aideck.c and CPX from the build. The old CPX protocol will no longer work as communication between STM and GAP8.

```bash
make clean
make
```

The generated firmware artifacts are written to `build/`.

## Generate TinyMPC firmware parameters

The controller consumes a fixed-size specialization generated from the current
upstream TinyMPC checkout. `TinyMPC` at the repository root must point to that
checkout, and Python must have `autograd` and `numpy` installed.

From `apps/controller_tinympc_eigen`, run:

```bash
python3 tools/firmware_codegen/tinympc_to_crazyflie_adapter.py
make
```

The adapter prompts for the Crazyflie type, attached decks, propeller guards,
horizon, model timestep, solve rate, ADMM iteration cap, half-space capacity,
and constrained portion of the horizon. Press Enter to accept any displayed
default. It then:

1. builds and linearizes the selected Crazyflie dynamics with Autograd;
2. calls current TinyMPC through `tools/firmware_codegen/tinympc_cpp_bridge.cpp` to precompute
the Riccati and affine caches; and
3. writes `src/tinympc_generated_params.h` for the STM32 build.

The generated header is the source of truth for the runtime horizon, prediction
timestep, solve rate, iteration cap, A/B/f model, Q/R costs, bounds, hover input,
and solver caches. Do not edit it manually. Regenerate it whenever the vehicle
configuration or solver parameters change, then rebuild and check the reported
flash/RAM use before flying. The embedded half-space store supports at most 25
horizon knots; the adapter rejects larger configurations.

## Flash

```bash
Expand All @@ -23,4 +55,100 @@ CLOAD_CMDS="-w radio://0/80/2M/E7E7E7E7E7" make cload

- The controller uses the Crazyflie out-of-tree app/controller build flow.
- `en_traj` in `src/controller_tinympc.cpp` is a compile-time flag, not a runtime Crazyflie parameter.
- On `main`, `en_traj` defaults to `false`, so the controller follows commander/setpoints unless you explicitly turn the baked trajectory back on in source.
- On `main`, `en_traj` currently defaults to `true`, so the compiled trajectory
is active after the controller handoff. Set it to `false` in source when the
controller should follow commander/setpoints instead.

### Local MPC frame

Each solve uses a local NWU frame anchored at the current position and yaw.
States, references, and obstacle planes are transformed into this frame. The
ordinary racing path sends TinyMPC's first optimized input directly to the
motors. Acrobatic stored-LTV builds do the same; there is no secondary
attitude, rate, or position controller.
Trajectory headers store:

```text
[p_W(3), q_WB(wxyz), v_W(3), omega_B(3)]
```

The quaternion is converted to local Rodrigues coordinates for the 12-state
MPC. Reference yaw is derived and unwrapped online from the quaternion sequence.
Regenerate the headers with:

```bash
python3 tools/firmware_codegen/generate_crazyflie_trajectory.py
```

### Acrobatic TinyMPC build

The roll and flip trajectories use a reference-centered quaternion-error chart,
stored per-motor command feedforward, and horizon-wise offline LTV matrices.
The LTV state is the 12-state tracking error plus four normalized rotor-RPM
errors. It includes the measured 64.9 ms rotor response, RPM-to-thrust/torque
curves, and the firmware's nominal Crazyflie inertia. TinyMPC runs at
50 Hz with the usual 20-knot horizon and five ADMM iterations, then sends all
four optimized motor commands directly to PWM. No online relinearization,
Riccati pass, or high-rate corrective cascade is used.

```bash
python3 tools/pybullet_simulation/generate_acrobatic_trajectories.py \
--firmware-header-dir src/trajectories/50hz
make TINYMPC_TRAJECTORY=roll_flip_360 TINYMPC_STORED_LTV=1
# Substitute front_flip_360, backflip_360, or barrel_roll_forward_360.
```

Regenerate one fully offline trajectory-linearized model sequence with:

```bash
python3 tools/pybullet_simulation/generate_stored_ltv.py \
sim/trajectories/acrobatics/roll_flip_360.csv \
--out sim_runs/stored_ltv_models/roll_flip_360.npz \
--firmware-header src/trajectories/50hz/ltv/stored_ltv_roll_flip_360_50hz.h
make TINYMPC_TRAJECTORY=roll_flip_360 TINYMPC_STORED_LTV=1
```

Omitting `TINYMPC_STORED_LTV=1` deliberately selects the original 12-state
fixed hover matrices and is retained as a negative-control build.

The generator performs nonlinear differentiation and the time-varying Riccati
pass on the host. Firmware only indexes stored `A/B/affine/P_affine/K/Hinv` data; it
never relinearizes, factors a Hessian, or runs a Riccati recursion onboard.

Run the stressed headless validation before producing firmware:

```bash
python3 tools/pybullet_simulation/run_acrobatic_suite.py \
--out sim_runs/tier3_tinympc_suite --controller tinympc-acro \
--profile realistic-v1 --overwrite
```

Plain `make` still selects the conservative circle trajectory. The acrobatic
build is experimental and must be validated on a restrained test stand before
free flight. A current stationary-flip image uses about 79% flash, 87% RAM,
and 85% CCM; the longer barrel-roll image is close to the flash limit.

### Independent CrazySim/MuJoCo validation

For a second physics implementation that executes this controller inside the
Crazyflie firmware SITL, use the repository-local CrazySim harness:

```bash
tools/crazysim_mujoco/run.sh \
--trajectory backflip_360 \
--stored-ltv 1 \
--out apps/controller_tinympc_eigen/sim_runs/crazysim/backflip_ltv \
--overwrite
```

It pins and patches its dependency under `tools/crazysim_mujoco/.deps`, builds
in Docker, and writes MuJoCo ground truth, controller logs, crash metrics, and a
four-panel validation plot. The harness couples the POSIX firmware tick to the
measured physics rate and rejects timing-invalid runs. The revised backflip
passed three clock-valid clean repetitions plus individual sensor-noise and
Flow-deck trials. A severe combined plant/estimator/weather stress completes
without contact from 2.0 m, although its lateral drift remains too large for
racing. The fixed hover-matrix negative control crashes without completing the
rotation. See
`docs/crazysim_direct_motor_robustness_2026-08-19.md` for the results and
`tools/crazysim_mujoco/README.md` for the fidelity boundary.
Loading