diff --git a/README.md b/README.md index b79b4a08f..ec74b1be2 100644 --- a/README.md +++ b/README.md @@ -2,46 +2,138 @@ [![Validate](https://github.com/existential-engineering/catalog/actions/workflows/validate.yml/badge.svg)](https://github.com/existential-engineering/catalog/actions/workflows/validate.yml) [![Release](https://github.com/existential-engineering/catalog/actions/workflows/release.yml/badge.svg)](https://github.com/existential-engineering/catalog/actions/workflows/release.yml) -[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) -[![Data: CC0](https://img.shields.io/badge/Data-CC0%201.0-blue.svg)](https://creativecommons.org/publicdomain/zero/1.0/) +[![Latest Release](https://img.shields.io/github/v/release/existential-engineering/catalog?label=database)](https://github.com/existential-engineering/catalog/releases/latest) +[![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/existential-engineering/catalog/badge)](https://scorecard.dev/viewer/?uri=github.com/existential-engineering/catalog) +[![License: MIT](https://img.shields.io/badge/code-MIT-yellow.svg)](LICENSE) +[![Data: CC0](https://img.shields.io/badge/data-CC0%201.0-blue.svg)](data/LICENSE) +[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](CONTRIBUTING.md) + +**An open, community-driven database of music production gear** — synths, +plugins, DAWs, audio interfaces, pedals, microphones, sample packs, and the +cables that connect them all. + +Over **12,000 products** from more than **4,000 manufacturers**, maintained as +human-readable YAML, validated by CI, and shipped as a ready-to-query SQLite +database on every release. The data is public domain (CC0), so you can build +whatever you want with it. + +## Why this exists + +Music production data is scattered across marketing pages, forum posts, and +manuals that go offline. This repo collects it in one structured, +version-controlled place: + +- **Structured** — strict schemas (Zod-validated) for every entry type, with + controlled vocabularies for categories, plugin formats, platforms, and + physical I/O connections. +- **Deep** — hardware entries model every physical jack (signal flow, connector + type, panel position), software entries carry bundle identifiers per plugin + format, and product generations are linked (`supersedes` chains from MKI to + MKII). +- **Verifiable** — every release ships `catalog.sqlite` with a SHA-256 + checksum, a minisign signature, and an SPDX SBOM. + +It powers [Aureo](https://aureo.audio), a desktop app for organizing your +studio — but it isn't tied to it. Plugin managers, gear-matching tools, +research datasets: the data is yours. + +## What's inside + +| Collection | Entries | What lives there | +| --------------- | ------: | ------------------------------------------------------- | +| `manufacturers` | 4,000+ | Companies and developers, including defunct brands | +| `hardware` | 6,100+ | Synths, interfaces, pedals, mixers, mics — with full I/O | +| `software` | 4,100+ | Plugins, DAWs, and standalone apps | +| `content` | 1,600+ | Preset packs, sample libraries, expansions | +| `accessories` | 700+ | Cables, stands, acoustic treatment | + +…and counting. Entries also support prices, release-history versions, videos, +search terms, translations, and product-lineage links. + +## Get the data + +Grab the latest build from +[GitHub Releases](https://github.com/existential-engineering/catalog/releases/latest): -Open source database of audio software, plugins, DAWs, and hardware for the music production community. +```bash +curl -LO https://github.com/existential-engineering/catalog/releases/latest/download/catalog.sqlite -## Overview +# Verify the checksum (optional but encouraged) +curl -LO https://github.com/existential-engineering/catalog/releases/latest/download/catalog.sqlite.sha256 +sha256sum -c catalog.sqlite.sha256 +``` -This repository contains: +Each release also includes a `catalog.sqlite.minisig` signature and a +`catalog-sbom.spdx.json` SBOM. -- **YAML Source Files** — Human-readable and version-controlled data files -- **SQLite Database** — Pre-built database generated from YAML (distributed via GitHub Releases) -- **Validation Scripts** — Ensures data integrity and schema compliance -- **CI/CD Automation** — Auto-builds and releases on changes +### Query it -## Data Structure +The database has full-text search (FTS5) indexes across every collection: -```text -data/ -├── manufacturers/ # Companies and developers -│ └── xfer-records.yaml -├── software/ # Plugins, DAWs, and standalone apps -│ ├── serum.yaml -│ └── ableton-live.yaml -└── hardware/ # Audio interfaces, controllers, instruments, etc. - └── apollo-twin-x.yaml +```sql +-- Full-text search across names, manufacturers, and search terms +SELECT s.name, m.name AS manufacturer +FROM software_fts f +JOIN software s ON s.id = f.id +JOIN manufacturers m ON m.id = s.manufacturer_id +WHERE software_fts MATCH 'wavetable'; + +-- Every audio interface in the catalog +SELECT h.name, m.name AS manufacturer +FROM hardware h +JOIN manufacturers m ON m.id = h.manufacturer_id +WHERE h.primary_category = 'audio-interface'; + +-- Plugin formats and bundle identifiers for a product +SELECT sf.format, sf.identifier +FROM software s +JOIN software_formats sf ON sf.software_id = s.id +WHERE s.name = 'Serum'; ``` -## YAML Format +## Contributing + +**Every entry in this catalog was added by someone who cared about getting the +details right.** Spotted a missing plugin? A synth with the wrong I/O? A typo +in a description? That's your opening — contributions of any size are welcome. + +```bash +# 1. Fork and clone, then install +pnpm install + +# 2. Add or edit YAML files under data/ -### Manufacturer +# 3. Validate (CI runs the same checks) +pnpm validate + +# 4. Open a pull request +``` + +Validation is your safety net: it checks schemas, cross-references, controlled +vocabularies, and formatting, and its error codes are documented in +[`docs/VALIDATION_ERRORS.md`](docs/VALIDATION_ERRORS.md). You don't need to +memorize the rules — run `pnpm validate` and it will tell you what's off. + +See [CONTRIBUTING.md](CONTRIBUTING.md) for the full guide, and +[`CLAUDE.md`](CLAUDE.md) for the authoritative field conventions. JSON Schemas +for editor autocomplete live in [`schema/json/`](schema/json/). + +### Entry format at a glance + +Filenames are slugs. A product file is named +`-.yaml`, and its `manufacturer` field +references the manufacturer's filename: ```yaml -slug: xfer-records +# data/manufacturers/xfer-records.yaml name: Xfer Records -url: https://xferrecords.com +url: https://xferrecords.com/ +description: Audio software company founded by Steve Duda, best known for + creating Serum, one of the most popular wavetable synthesizers. ``` -### Software (Plugin) - ```yaml +# data/software/xfer-records-serum.yaml name: Serum manufacturer: xfer-records primaryCategory: synthesizer @@ -56,123 +148,78 @@ platforms: - mac - windows identifiers: - au: com.xferrecords.Serum + default: com.xferrecords.Serum vst3: com.xferrecords.Serum.vst3 -url: https://xferrecords.com/products/serum -description: Advanced wavetable synthesizer with visual feedback +url: https://xferrecords.com/products/serum-2 +description: Serum is an advanced wavetable synthesizer by Xfer Records with + a visual, high-quality workflow. ``` -### Software (DAW) - -```yaml -name: Live -manufacturer: ableton -primaryCategory: daw -platforms: - - mac - - windows -identifiers: - bundle: com.ableton.live -url: https://ableton.com/live -``` - -### Hardware - ```yaml +# data/hardware/universal-audio-apollo-twin-x.yaml name: Apollo Twin X manufacturer: universal-audio primaryCategory: audio-interface url: https://www.uaudio.com/products/apollo-twin-x -description: Desktop Thunderbolt audio interface with UAD processing +description: Desktop Thunderbolt audio interface with realtime UAD processing. +io: + - name: Hi-Z Instrument Input + signalFlow: input + category: audio + type: instrument + connection: 1/4-inch + maxConnections: 1 + position: Bottom ``` -## Using the Database - -### Download Latest Release - -```bash -# Get the latest SQLite database -curl -L https://github.com/existential-engineering/catalog/releases/latest/download/catalog.sqlite -o catalog.sqlite -``` - -### Query Examples - -```sql --- Search for plugins by name -SELECT s.name, m.name as manufacturer -FROM software s -JOIN manufacturers m ON s.manufacturer_id = m.id -WHERE s.id IN ( - SELECT id FROM software_fts WHERE software_fts MATCH 'synth*' -); - --- Find plugins by category -SELECT s.name, m.name as manufacturer -FROM software s -JOIN manufacturers m ON s.manufacturer_id = m.id -JOIN software_categories sc ON s.id = sc.software_id -WHERE sc.category = 'synthesizer'; - --- Get plugin formats and identifiers -SELECT sf.format, sf.identifier -FROM software_formats sf -WHERE sf.software_id = 'serum'; - --- Find all DAWs -SELECT s.name, m.name as manufacturer -FROM software s -JOIN manufacturers m ON s.manufacturer_id = m.id -WHERE s.primary_category = 'daw'; -``` - -## Contributing - -We welcome contributions! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines. - -### Quick Start - -1. Fork the repository -2. Add or update YAML files in `data/` -3. Run validation: `pnpm validate` -4. Submit a pull request - -### Adding New Software - -1. Ensure the manufacturer exists in `data/manufacturers/` -2. Create a YAML file in `data/software/` with the slug as filename -3. Use categories from `schema/categories.yaml` -4. Use formats from `schema/formats.yaml` +Don't add an `id` field yourself — CI assigns a unique ID to new entries +automatically. Valid categories, formats, platforms, and I/O vocabularies live +in [`schema/`](schema/). ## Development ```bash -# Install dependencies -pnpm install +pnpm install # Install dependencies -# Validate all YAML files -pnpm validate - -# Validate only the files you touched: a fast pre-flight while editing. -# It skips cross-file checks (duplicate IDs, supersedes targets and -# cycles), so run the full `pnpm validate` above before committing. +pnpm validate # Validate all YAML data files pnpm validate --files data/hardware/some-entry.yaml + # Fast pre-flight for just the files you touched. + # Skips cross-file checks (duplicate IDs, supersedes + # targets), so run the full validate before committing. + +pnpm format # Format all YAML files (or pass specific paths) +pnpm build # Build catalog.sqlite locally (output in dist/) +pnpm typecheck # Type-check the scripts +pnpm test # Run the script test suite +``` -# Format all YAML files, or only the ones you touched -pnpm format -pnpm format data/hardware/some-entry.yaml - -# Build SQLite database locally -pnpm build +## Repository layout -# Type check -pnpm typecheck +```text +data/ +├── manufacturers/ # Companies and developers +├── software/ # Plugins, DAWs, standalone apps +├── hardware/ # Interfaces, synths, pedals, mics… +├── content/ # Presets, sample packs, expansions +└── accessories/ # Cables, stands, acoustic treatment +schema/ # Controlled vocabularies + generated JSON Schemas +scripts/ # Validation, build, and maintenance tooling +docs/ # Validation error reference, schema versioning +dist/ # Built SQLite output (local builds) ``` +Every push to `main` re-validates the dataset, and releases are cut +automatically through [Changesets](https://github.com/changesets/changesets): +when the pending version PR merges, CI rebuilds `catalog.sqlite`, signs it, +and publishes it to GitHub Releases. + ## License -- **Code** (scripts, CI configs): [MIT License](LICENSE) -- **Data** (YAML files): [CC0 1.0 (Public Domain)](data/LICENSE) +- **Code** (scripts, CI configs): [MIT](LICENSE) +- **Data** (everything under `data/`): [CC0 1.0 — public domain](data/LICENSE) -## Related Projects +## Related -- [Aureo](https://aureo.audio) — Studio organization app that uses this catalog +- [Aureo](https://aureo.audio) — studio organization app built on this catalog +- [Releases](https://github.com/existential-engineering/catalog/releases) — + versioned database builds with checksums and signatures