Skip to content

development.md

Latest commit

 

History

History
255 lines (211 loc) · 12.1 KB

development.md

File metadata and controls

255 lines (211 loc) · 12.1 KB
Raw

Working on MPVST itself

Building the plug-in from source, running its gates, and cutting a release. To write instruments or effects for MPVST - which needs none of this - see writing-scripts.md.

Repository layout
src/ the C++ that builds the plug-in: plugin/ (VST3 classes), protocol/ (the shared-memory wire format), runtime/ (shared memory and child processes)
usermods/ the MicroPython C modules the engine binds to: vstaudio/ (the audio API scripts use) and vstui/ (the editor's framebuffer, input and edit rings)
lib/ everything staged into the bundle beside the engine: the bootstrap, the adapters, mpvst_scan_plugins.py, the default instrument, and the editor's Python half (mpvst_editor.py, mpvst_board_config.py, mpvst_panel/)
tools/ developer tooling - the library test sweeps, audio_qc.py, derive_patches.py
tests/ the ctest suite and smoke_host/, a minimal VST3 host that loads the bundle with no DAW
scripts/ build, packaging and setup automation
installer/ what a user runs rather than builds: windows.nsi (the NSIS installer, cross-built from Linux) and install-linux.sh, which ships inside the tarball as install.sh
reaper/ + reaper.sh everything that drives REAPER. Deletable as a unit; nothing outside it depends on it
examples/ two composers with their songs, and bounce.py; nothing in here imports anything above it (rendering: rendering.md)

The architecture is written down in docs/architecture/phase-0.md (the system boundary and what each process owns) and docs/architecture/ipc-v1.md (the shared-memory protocol rules), with docs/architecture/ui-v1.md covering the editor. ipc-v1.md and ui-v1.md describe the shipping design; phase-0.md is historical - the Windows-only, no-editor design accepted before Linux shipped and before the LVGL editor existed - kept for its still-valid process/thread/state reasoning, not as a description of what ships today. The canonical structure sizes and offsets live in src/protocol/include/mpvst/protocol.h and src/protocol/include/mpvst/ui.h.

Prerequisites

  • A C++17 toolchain: MSVC on Windows, GCC or Clang on Linux. The Windows plug-in binary is the one thing here that needs MSVC, and the full Visual Studio IDE is not required - Visual Studio Build Tools with the Desktop C++ workload and a Windows SDK is enough, and is about half the disk:

    winget install --id Microsoft.VisualStudio.BuildTools --override ^
  "--quiet --wait --norestart --add Microsoft.VisualStudio.Workload.VCTools ^
   --add Microsoft.VisualStudio.Component.Windows11SDK.26100 --includeRecommended"

    The generator the install script names is Visual Studio 18 2026, so it wants the 2026 Build Tools (18.x); the winget id above has no year in it and resolves to that. Nothing else in the repository needs it: the Windows sidecar engine is cross-compiled from WSL with MinGW, the Linux build uses GCC, and a Python-only change to the instrument or effect library never touches a compiler at all (see the staging note).

  • CMake 3.25+ and Ninja (cmake -S . -B .build-linux -G Ninja is the documented invocation below).

  • On Linux, X11 development headers (src/plugin/CMakeLists.txt runs find_package(X11 REQUIRED) for the editor's native window) - e.g. libx11-dev on Debian/Ubuntu.

  • Python 3.x for tools/, scripts/, and the ctest-registered Python suites; numpy, pydevices-audioif, pydevices-audioinstruments and pydevices-audioeffects (all from TestPyPI - see tools/README.md) for the instrument/effect tests and preview renders, and flake8 for the mpvst_lint ctest. scripts/bootstrap.sh creates a repo-local .venv with these.

  • The Steinberg VST3 SDK, fetched by scripts/fetch-vst3-sdk.sh into the gitignored .deps/vst3sdk (see the licence note for its terms).

  • The sibling audioif checkout and the org's optional build-aggregator workspace the MicroPython engine build depends on, and the sibling audiocomponents checkout whose audioinstruments and audioeffects packages the plug-in build stages into the bundle - all three fetched by scripts/fetch-sibling-repos.sh.

  • On WSL, building the Windows engine/plugin needs a reachable Windows host: scripts/build-micropython-engine.sh --port windows and scripts/install-plugin-windows.sh both shell out to powershell.exe, and scripts/bootstrap.sh skips the Windows engine port automatically when /mnt/c/Users or powershell.exe is not available.

Getting started

A fresh clone has none of the external dependencies this repo needs - the VST3 SDK, the sibling audioif and build-aggregator repos the engine build depends on, the sibling audiocomponents repo the plug-in build stages its instruments and effects from, or REAPER for the DAW-driven tooling. .deps/ and those sibling checkouts are all gitignored. One command sets all of it up:

./scripts/bootstrap.sh

See scripts/README.md for what it does and how to run each step individually.

Building and testing

The MicroPython sidecar is built separately from the plug-in, and only needs rebuilding when usermods/vstaudio, usermods/vstui, or the sibling audioif checkout's C sources change. It lands in the ignored .deps/engine/, and the plug-in build stages it into the bundle. CMake never detects a stale engine on its own - it only re-stages the file at MPVST_MICROPYTHON_ENGINE if that path's mtime changes, so after any of those three changes you must rerun the build script yourself before reconfiguring/rebuilding the plug-in:

./scripts/build-micropython-engine.sh --port windows
./scripts/build-micropython-engine.sh --port unix

Linux:

cmake -S . -B .build-linux -G Ninja
cmake --build .build-linux
ctest --test-dir .build-linux --output-on-failure

Windows, driven from WSL with the vendored CMake. scripts/install-plugin-windows.sh wraps the build and installs the result into the per-user VST3 directory a DAW scans:

./scripts/install-plugin-windows.sh

A Python-only change needs no compiler

Adding an instrument to audiocomponents, editing a script, or changing what the catalog says does not touch the plug-in binary. If the Windows build directory is gone - it lives under %LOCALAPPDATA%\Temp, which Windows cleans - you do not have to reconfigure MSVC to get the change into a DAW. Stage the packages into the installed bundle and rewrite the two metadata files with the bundle's own engine:

B="$WIN_LOCALAPPDATA/Programs/Common/VST3/MPVST.vst3/Contents/x86_64-win"
for pkg in audioinstruments audioeffects; do
    rm -rf "$B/$pkg" && mkdir -p "$B/$pkg"
    (cd ../audiocomponents/lib/$pkg && tar -cf - --exclude=__pycache__ \
        --exclude='*.egg-info' --exclude=pyproject.toml .) | (cd "$B/$pkg" && tar -xf -)
done
(cd "$B" && ./mpvst-engine.exe -X heapsize=64M mpvst_scan_plugins.py \
         && ./mpvst-engine.exe -X heapsize=64M mpvst_catalog.py)

That is what cmake --build does for these two directories, minus the compiler: the excludes match src/plugin/stage_lib.cmake, and the two scripts rewrite moduleinfo.json and catalog.json. A new instrument does not need a REAPER rescan either - the soundtrack composer's projects load the generic "MPVST Script Host" class with the script embedded in state, so the only thing REAPER has to find is the staged package the sidecar imports.

Reconfiguring the Windows build

install-plugin-windows.sh refuses to run without a configured build, which is what you see after the Temp directory is cleaned or MSVC is reinstalled:

source scripts/windows-paths.sh && mpvst_load_windows_paths
'.deps/cmake-4.4.2-windows-x86_64/bin/cmake.exe' \
    -S "$(wslpath -w .)" -B "$(wslpath -w "$WIN_TEMP/mpvst-build")" \
    -G 'Visual Studio 18 2026'

The Linux CMake cache remembers the engine path. After switching engines, reconfigure with cmake -S . -B .build-linux -U MPVST_MICROPYTHON_ENGINE. It remembers MPVST_AUDIOIF_LIB the same way - the name the components path had while the packages lived in audioif, still honoured for one release with a warning - so a build directory configured before the move keeps staging from audioif until you reconfigure with -U MPVST_AUDIOIF_LIB.

Steinberg hosting tools are off by default so a plug-in-only build does not pull in editor-host dependencies. Enable them in a dedicated validator build with -DSMTG_ENABLE_VST3_HOSTING_EXAMPLES=ON. VST3_SDK_ROOT may point at an existing SDK checkout instead of the fetched one.

Building a release

VERSION at the repository root is the single source of truth - CMake and both packaging scripts read it, and editing it re-runs CMake's configure step, so a binary and the archive around it cannot disagree about which version they are.

./scripts/fetch-nsis.sh          # once, for the Windows installer
./scripts/package-linux.sh
./scripts/package-windows.sh

Each produces a versioned archive plus a SHA-256 sidecar under the ignored dist/, after verifying the bundle carries its engine and bootstrap; package-windows.sh also builds the installer, from the same staging tree the archive is made from, so the two cannot ship different bytes. The installer is built by NSIS, which cross-builds a Windows installer from Linux - fetch-nsis.sh unpacks it into .deps/ rather than installing it on the machine, so it needs no root and removing .deps removes it. See docs/windows/README.md and docs/linux/README.md for the development install paths and the desktop-script security model.

This repository deliberately has no hosted CI. The 14-test ctest suite (lint included) is the gate, and it is run locally - by a developer before pushing, or by scripts/bootstrap.sh as its final verification step. Hosted CI is planned to arrive with the post-program refactor, not before.

Testing against a real DAW

ctest covers the plug-in with no DAW involved. Two further harnesses use REAPER, and both need the packaged plug-in installed first because they exercise the installed bundle:

./reaper/matrix/run-reaper-matrix.sh --platform windows
./reaper/matrix/run-reaper-matrix.sh --platform linux

The matrix drives REAPER headlessly through a startup ReaScript, covering what only a real host can - FX chain add/remove, parameter automation, project save/reload, macro resync. It overwrites Scripts/__startup.lua in REAPER's resource path, so remove that file before using REAPER interactively. A host with no live audio device only processes during a render, so the matrix forces a short render before reading any status parameter.

./scripts/check-cross-platform-parity.sh

Both smoke hosts render a fixed score through the real MicroPython sidecar and the raw float32 PCM is compared. The current result is an identical SHA-256 - the platforms agree exactly, not within a tolerance.

./reaper.sh renders and plays the example pieces; see reaper/README.md and examples/soundtrack/README.md.

Workspace isolation

The sibling audioif and audiocomponents repositories are consumed read-only - no build or formatting command here writes into either. The engine builder likewise leaves the sibling MicroPython checkout unchanged: it uses the build workspace's existing transactional overlay, and removes the temporary vstaudio module link on exit, including after a failed build.

Deferred

  • Effect extras: a wet/dry mix parameter and sidechain input buses.
  • Float64 host processing and a native floating-point audioif graph.
  • macOS bundles, signing, notarisation, and universal binaries.
  • Coverage-guided fuzzing. tests/fuzz exposes libFuzzer entry points; configure with -DMPVST_ENABLE_LIBFUZZER=ON on a clang toolchain and keep interesting inputs in tests/fuzz/corpus. The portable driver runs on every toolchain as an ordinary test regardless.