RetroBoxDB MSX2

English | 中文说明

Single-file SQLite preservation database for Microsoft MSX2. The public Catalog holds metadata only (checksums, DAT and provenance records, header fields, archive recipes and the processing code); it contains no ROM data and cannot restore files. The populated database stays local.

Item	Value
Original size	293 source ZIPs, 50.6 MiB (No-Intro 204, RetroAchievements sets 89); 321 ROM files, 111.1 MiB uncompressed
Stored size	populated database 30.5 MiB; public Catalog 3.5 MiB (no ROM data)
Ratio	60.2% of the source ZIPs, 27.4% of the uncompressed ROM files
Technology	storage v4: SHA256-deduplicated 128 KiB blocks packed in No-Intro family order into solid LZMA2 groups of up to 128 MiB (128 MiB dictionary); per-block SHA256 and per-object CRC32/MD5/SHA1/SHA256 verification; source ZIPs reproduced byte-for-byte from TorrentZip plans
Export performance	Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz, idle, Python 3.14.4, all checks included. whole newest-DAT set with export_set.py (201 files, each checked against the DAT hashes): 30.7 MiB/s, 8 ms per file on average; single file with a cold cache (the group is decoded up to the file): ROM 0.399 s, TorrentZip 0.473 s on average

Downloads and documents

File / document	Content
RetroBoxDB.MSX2.Catalog.sqlite	Public Catalog (Release asset with SHA256SUMS)
Storage v4 guide / 中文	Storage evaluation, contents, RA, names and maintenance for every platform
Technical design	Storage format, platform adapters, incremental updates, verification
RA list / summary, build report, audit resolution	Detailed data

Storage choice and platform specifics

21 block/group combinations measured on the whole local collection (assessment/data/storage-experiment-msx2.json): smallest 1024 KiB / 128 MiB at 24.45 MiB; by the rule (within 0.5% of the smallest, the smallest block, then the smallest group) 128 KiB / 128 MiB at 24.50 MiB. ZIPs 50.62 MiB, per-file LZMA 40.50 MiB.

• Cartridges, floppy disks and tapes: msx_hardware.media records the medium; cartridges carry the AB header at 0x0000 or 0x4000 (INIT, STATEMENT, DEVICE, TEXT), disks are recognised by size and boot sector, tapes by the CAS block magic. The machine generation is not in the data; mappers need external evidence.
• The RetroAchievements MSX folder (console 29) mixes MSX and MSX2 games. It is split platform first, medium second (tools/msx_route.py): each ZIP goes to exactly one database by a member SHA1 in the MSX2 / MSX DATs, an (MSX2 tag or .mx2 member, the reviewed list data/msx-routing.csv (reference lists, public references such as MSX software databases and contest pages, manual knowledge), a title only one platform's DATs name, or at least 10% of 8 KiB blocks shared with one platform's DAT ROMs (hacks, translations). Of its 200 ZIPs, 89 are in this database and 111 in RetroBoxDB-MSX; none is unconfirmed. The basis of each file is in rom_annotations (kind='platform') and v_msx_headers.platform_evidence.
• RetroAchievements console 29 is shared with the other MSX database; the RA report covers only games tied to this database.

Contents

Item	Value
ROM records / games / releases	295 / 157 / 201
DAT coverage per version	20260124-112728: 201/201
Local ROMs in no DAT	94
ROM files of the RetroAchievements set	in a No-Intro DAT 23, RA only 93, hash not in the latest RA snapshot 1 (list); RA games still without a local ROM: gap list
No-Intro DB Export + Dump Log 20260124-112728	201 archives, 204 file identities, 2 documented hardware assertions; Dump Log Verified 0
RetroAchievements (console 29)	45 games with achievements: 45 with a local ROM (116 ROMs), 0 with the ROM in a sibling database, 0 DAT only, 0 DB file only, 0 without a No-Intro counterpart
Chinese names	198 of 198 rows translated (154 unique); 198 local ROMs have a Chinese name
Populated-database audit	306 objects, 2 groups, 277 archive plans, all passed

Every source ZIP is reproduced byte-for-byte from its TorrentZip plan (v_file_checksums.exported_bytes_equal_source).

Usage

# Query-only audit with the Catalog's embedded engine (also: stats, checksums FILE_ID, help)
python3 -B -c 'import sqlite3,sys; c=sqlite3.connect(sys.argv[1]); s=c.execute("SELECT content FROM resources WHERE name=?",("engine.py",)).fetchone()[0]; c.close(); exec(compile(s,"RetroBoxDB:engine.py","exec"))' ./RetroBoxDB.MSX2.Catalog.sqlite audit
# Populated database: export by DAT version, 1G1R, RA achievements, TorrentZip or plain ROMs
python3 -B tools/export_set.py RetroBoxDB.MSX2.sqlite OUT --set 1g1r --ra achievements --container torrentzip --layout ra-category
# Add new DATs, DB Export / Dump Log snapshots, ROMs and an RA snapshot incrementally
python3 -B tools/update_db.py RetroBoxDB.MSX2.sqlite --discover --ra --catalog RetroBoxDB.MSX2.Catalog.sqlite

Python 3.10+ standard library only. engine.py and the other resources entries are executable code; run them only from a database you built or a Release asset whose SHA256 you verified. Releases are produced by .github/workflows/publish-catalog.yml: it starts from the base Catalog pinned in release/catalog-release.json, injects the engine and documents of this commit, checks every data-table digest, runs the tests and the audit, then publishes.