diff --git a/README.md b/README.md
index 62152f34d..3fcd49996 100644
--- a/README.md
+++ b/README.md
@@ -181,11 +181,11 @@ To undo it, remove the entries in Windows Security → App & browser control →
| **Video**⁵: MP4, MOV, MKV, WEBM, AVI, WMV, FLV, MPEG, Video CD (DAT), TS, M2TS, 3GP, OGV, VOB, …;
Wisenet WAVE / Nx Witness exports (NOV) read | MP4 (H.264, HEVC, AV1), WEBM, MOV, MKV, AVI, GIF, APNG, MP3, M4A, 720p, split into parts;
subtitle tracks kept (MKV, MP4, MOV, WebM);
subtitles (SRT, ASS, WebVTT) |
| **Audio**⁵: MP3, M4A, AAC, WAV, FLAC, OGG, OPUS, WMA, AIFF, AMR, APE, WavPack, TTA, DSD (DSF, DFF), DTS, AC3, E-AC3, …;
tracker modules (XM, IT, S3M) read | MP3, M4A (AAC, ALAC), M4R (iPhone ringtone), WAV, FLAC, OGG, OPUS, AAC, WavPack, TTA, W64, AC3, E-AC3 |
| **Subtitles**: SRT, ASS/SSA, WebVTT, SAMI (SMI), LRC, KRC, TTML, MicroDVD, SCC | each other as UTF-8 (Korean CP949 files read correctly) |
-| **Archives**: ZIP, ANSYS Workbench (WBPZ), 7Z, RAR, TAR, TAR.GZ, TAR.BZ2, TAR.XZ, GZ, ISO, CAB, DMG, ALZ, EGG, …;
JAR, NuGet (NUPKG), Android packages (APKS, XAPK, APKM, AAB), Adobe extensions (ZXP) read | extract, ZIP, 7Z, TAR, TAR.GZ;
any files → one ZIP; folders → ZIP, ISO (Joliet or UDF 2.01) |
+| **Archives**: ZIP, ANSYS Workbench (WBPZ), 7Z, RAR, TAR, TAR.GZ, TAR.BZ2, TAR.XZ, GZ, ISO, CAB, DMG, ALZ, EGG, Garry's Mod addon (GMA), Valve pak (VPK), files packed into a Source map (BSP), …;
JAR, NuGet (NUPKG), Android packages (APKS, XAPK, APKM, AAB), Adobe extensions (ZXP) read | extract, ZIP, 7Z, TAR, TAR.GZ, GMA;
any files → one ZIP; folders → ZIP, ISO (Joliet or UDF 2.01), GMA |
| **CAD**: DWG, DXF
Gerber PCB layers (GBR, GTL, GBL, GTO, …) read;
G-code of 3D printers and CNC machines (GCODE, GCO, NGC) read | PDF, SVG, PNG, DWG ↔ DXF;
G-code → toolpath (from above or isometric) or the slicer's preview |
| **Fonts**: TTF, OTF, WOFF, WOFF2, EOT;
FON (Windows bitmap fonts) read | each other;
FON → one font per size |
| **Data**: JSON, JSON Lines, YAML, TOML, XML
vCard, iCalendar | each other, CSV and XLSX (tables);
vCard, iCalendar ↔ CSV |
-| **3D models**: GLB, glTF, STL, OBJ (+ MTL), 3MF, PLY;
FBX read | each other, PNG preview |
+| **3D models**: GLB, glTF, STL, OBJ (+ MTL), 3MF, PLY, studiomdl data (SMD);
FBX, Source engine model (MDL, with its VTF textures) read | each other, PNG preview;
MDL → QC + SMDs (decompiled, with physics and animations) |
| **Geospatial vectors**: Shapefile, GeoJSON, KML, KMZ, GPX
CSV with coordinates; Garmin FIT, OpenStreetMap (.osm) read | each other and CSV / XLSX / JSON;
projected data (UTM, Web Mercator, Lambert, Albers) comes out as WGS 84 longitude / latitude |
Unmarked formats work out of the box, without Office, Hancom Office or LibreOffice.
diff --git a/docs/ENGINES.md b/docs/ENGINES.md
index 5b5d0b5d4..4556e693f 100644
--- a/docs/ENGINES.md
+++ b/docs/ENGINES.md
@@ -33,7 +33,7 @@ Filee chooses engines automatically. Settings → *Engines* shows their status,
| Pictures to 3D relief | built in | **PNG, JPG, WEBP, … (every picture ImageMagick reads) → GLB, glTF, STL, OBJ, 3MF, PLY**: a relief, brighter areas higher |
| **Geospatial vector data** | built in | **GeoJSON, KML, KMZ, GPX, Shapefile ↔ each other and CSV** (longitude / latitude or WKT); Garmin FIT (activities) and OpenStreetMap XML (`.osm`), both read only, → each of them and CSV; projected Shapefiles (UTM, Web Mercator, Lambert, Albers) are converted to WGS 84; through CSV to XLSX, JSON, … (see below) |
| rhwp | bundled with the installer (`engines/rhwp`, with the Microsoft C++ runtime, see below) | HWP/HWPX → PDF, **HWP → HWPX, HWPX → HWP** |
-| Archives (7-Zip) | bundled with the installer (`engines/7zip`, ~2.5 MB) + built-in readers | ZIP (including ANSYS Workbench WBPZ), 7Z, RAR, TAR (+ GZ/BZ2/XZ/Z/7Z/LZ), CAB, ISO, DMG, split (.001), Minecraft and Modrinth packs, … **→ folder, ZIP, 7Z, TAR, TAR.GZ/BZ2/XZ, the packs**; ALZ, EGG, lzip read in-process; "Compress into one archive" for any files |
+| Archives (7-Zip) | bundled with the installer (`engines/7zip`, ~2.5 MB) + built-in readers | ZIP (including ANSYS Workbench WBPZ), 7Z, RAR, TAR (+ GZ/BZ2/XZ/Z/7Z/LZ), CAB, ISO, DMG, split (.001), Minecraft and Modrinth packs, Garry's Mod addons (GMA), Valve paks (VPK), the files packed into Source maps (BSP), … **→ folder, ZIP, 7Z, TAR, TAR.GZ/BZ2/XZ, the packs, GMA**; ALZ, EGG, lzip, GMA, VPK read in-process; "Compress into one archive" for any files |
| LibreOffice + H2Orestart + Java | **optional download** (~420 MB, 1.3 GB on disk) | older and rare formats only: DOC, XLS, PPT, RTF and OpenDocument → PDF and to each other, output as DOCX/ODT/ODS/ODP; HWP/HWPX → DOCX/ODT; read-only formats (see below) → PDF and their kind's editable formats; FODT, OpenDocument templates (OTT, OTS, OTP) and XLT out; SVG → EMF/WMF/ODG, a PDF page → SVG |
| Pandoc | **optional download** (~42 MB, 240 MB on disk) | Markdown ↔ DOCX/ODT/RTF, DOCX/ODT/HTML/RTF → Markdown, HTML ↔ DOCX/ODT, markup languages (reStructuredText, LaTeX, Org, AsciiDoc, Typst, MediaWiki, DocBook, Jupyter, Textile, man, OPML) ↔ Markdown/HTML/DOCX/ODT and each other (and → RTF, EPUB, TXT), Markdown and markup → PPTX slides |
| Ghostscript | **optional download** (~20 MB, 31 MB on disk) | EPS/PS → PDF, PDF → EPS/PS, PostScript-based AI → PDF (images through PDF and PDFium); smaller PDFs when a preset asks for it; PDF/A-2b for PDF/A presets; renders CMYK PDF/AI pages for CMYK JPG/TIFF |
@@ -1421,8 +1421,8 @@ to draw). G-code is a read-only CAD format (fields: 3D models and CAD).
## 3D models
-`Filee.Engines/Models3D` converts GLB, glTF, STL, OBJ, 3MF and PLY into each other, and FBX into any of them, in
-managed code: every reader
+`Filee.Engines/Models3D` converts GLB, glTF, STL, OBJ, 3MF and PLY into each other, FBX and Source engine models (MDL)
+into any of them, and studiomdl data (SMD) to and from them, in managed code: every reader
fills one in-memory model (`Model3D`: parts with positions, normals, UVs, vertex colours and triangles; materials
with a base colour and a base colour texture) and every writer takes it, so each pair is one step. glTF and GLB are
read and written with [SharpGLTF](https://github.com/vpenades/SharpGLTF) (MIT); STL, OBJ/MTL, PLY, 3MF and the FBX
@@ -1456,6 +1456,8 @@ What each format keeps:
| OBJ + MTL | `o` / `g` / `usemtl` as parts, `v` (+ r g b), `vt`, `vn`, faces of any size; `mtllib` with `Kd`, `d` / `Tr`, `map_Kd` (options skipped) next to the file | **the `.mtl` and the textures are written next to the `.obj`** (`name.obj`, `name.mtl`, `name_texture.png`, named like the output); vertex colours as `v x y z r g b` |
| 3MF | objects, components (also in other model parts, as Bambu Studio writes them), build transforms, base materials, colour groups (materials extension); `support` / `other` objects skipped | one object per part, base materials from material colours, a colour group from vertex colours |
| FBX (read only) | binary 6.x–7.x (64-bit records from 7.5, zlib-compressed arrays) and ASCII; mesh geometry with polygons of any size, the first normal, UV, colour and material layer (by polygon vertex, control point or polygon, or all the same; direct or indexed); the model hierarchy from the connections with translation, rotation (every Euler order, pre- / post-rotation, pivots and offsets), scaling and geometric transforms; property templates from Definitions; material `DiffuseColor` and `Opacity`; the diffuse texture embedded (Video `Content`) or named next to the file; animation, skins, blend shapes, NURBS, cameras and lights skipped (the rest pose is read) | — (not written) |
+| SMD (studiomdl data) | the triangles of a reference SMD, one part per material (bones and weights not kept) | from an MDL: its bones, bind pose and weights (see below); from the others: a static prop with one root bone |
+| MDL (Source engine, read only) | versions 44–49 with the `.vvd` and `.dx90.vtx` (else `.vtx`, `.dx80.vtx`, `.sw.vtx`) beside it (see below) | — (needs Valve's studiomdl) |
| PLY | ASCII, binary little / big endian; vertex position, normal, UV (`s`/`t`, `u`/`v`, `texture_u`/`texture_v`), colours of any type; face lists with per-corner `texcoord` and face colours; `comment TextureFile`; other elements skipped | binary little endian, one mesh; material colours become vertex colours |
Lost on the way (no place in the target): **STL** keeps only the triangles (no colours, UVs, normals or materials);
@@ -1470,6 +1472,67 @@ FBX's texture names) are read only from the model's folder or a folder below it.
points anywhere else is looked up by its name in the model's folder instead, so a downloaded model cannot pull a
picture from elsewhere on the disk into its output.
+**Source engine models** (`.mdl`, #1526, the Games field; `Models3D/SourceModel.cs`): Garry's Mod, Counter-Strike:
+Source, Team Fortress 2 and Half-Life 2 models are three files studiomdl writes side by side: the `.mdl` (bones,
+textures, texture folders, the skin table, body parts → models → meshes), the `.vvd` (vertices with up to three bone
+weights; fixups say which runs make LOD 0) and the `.dx90.vtx` (per mesh, strip groups of triangle lists or strips
+indexing the mesh's vertices). All three must have the `.mdl`'s checksum, else the files belong to different builds and
+the model is refused; a missing `.vvd` or `.vtx` is named. The model is read in its bind pose with its default body
+groups (the first model of each body part), the first skin and LOD 0; animations, flexes, attachments and the physics
+mesh (`.phy`) are not read. Source is Z up and measured in inches (25.4 mm), its triangles are clockwise and its UVs
+start at the top left, so the model is turned to Filee's Y up, millimetres, counter-clockwise and bottom-left UVs; each
+mesh becomes a part named after its model, with one material per texture. Later branches' 8-byte-longer strip groups
+(CS:GO) are read when the indices of the usual layout don't fit.
+The textures are found as Garry's Mod finds them: in the addon's or game's own layout, the `materials` folder beside the
+`models` folder the model is in, `materials//.vmt` for each of the model's texture folders
+(`$cdmaterials`), letter case aside, then the material's `$basetexture` (also through a `patch` material's `include`) as
+a `.vtf`, decoded by the VTF reader into a PNG texture; `$nocull` makes the material double-sided. Only `.vmt` and
+`.vtf` files below that `materials` folder are read, never through `..`. A model outside such a layout, or one whose
+materials come from the game itself (the HL2 textures), keeps plain materials. OBJ output writes the textures beside the
+`.obj`, GLB embeds them.
+
+**Studiomdl data** (`.smd`, #1526, the Games field; `Models3D/SmdFormat.cs`): the text Valve's studiomdl compiles models
+from (`nodes`, a `skeleton` at `time 0`, `triangles`: a material line and three vertex lines of parent bone, position,
+normal, UV and bone weights), Z up, in inches, counter-clockwise, UVs from the bottom left. An MDL becomes a reference
+SMD as Crowbar decompiles it: its bones with their bind pose (position and rotation relative to the parent), its
+triangles with up to three bone weights per vertex and the textures as materials, read straight from the model so the
+skeleton is kept (Filee's own 3D model has no place for it). Any other model becomes a static prop: one `root` bone,
+every vertex weighted to it, the material's (or texture's) name as the material line, normals computed where the model
+has none. Reading takes the triangles of a reference SMD as a static model; an animation SMD (no triangles) is refused
+with "no triangles", a broken line with its number. The "SMD" preset is in the palette, after glTF.
+
+**Decompiling Source engine models** (#1526, `Models3D/SourceDecompiler.cs`, with `SourceModelDetails`,
+`SourceAnimation` and `SourcePhysics`): the "QC + SMD" preset (in the palette after "SMD", settings schema 98; only a
+`.mdl` takes it) turns a model into a folder named after it with what studiomdl builds it from, as Crowbar decompiles:
+
+| File | Holds |
+|---|---|
+| `.qc` | in Crowbar's order and layout: `$modelname`, `$staticprop`, `$bodygroup`s, `$lod`/`$shadowlod` blocks, `$surfaceprop`, `$contents`, `$illumposition` and `$eyeposition` (turned back the quarter turn studiomdl gives them), the other flags (`$opaque`, `$mostlyopaque` …), `$cdmaterials`, `$texturegroup` (the textures that differ between skins), `$attachment`s, `$cbox`, `$bbox`, a `$definebone` per bone, `$hboxset`/`$hbox`es (not when studiomdl made them), `$poseparameter`s, `$sectionframes`, a `$sequence` per sequence (its animations, activity and weight, fade times, fps, `loop`/`snap`/`delta`/… options, motion flags, blend parameters, events), `$includemodel`s (without the `models/` studiomdl adds), `$collisionmodel` or `$collisionjoints` with its settings, `$keyvalues` |
+| `.smd` | one reference SMD per body group model with triangles (blank models are `blank` in the QC) |
+| `_lod.smd` | a model's triangles at LOD *n*, written only when they differ from the LOD before; a model a LOD drops is `removemodel` |
+| `_physics.smd` | the convex pieces of the `.phy`, every vertex on its piece's bone (material `phy`) |
+| `_anims/.smd` | every frame of every bone of each animation |
+
+LODs use the LOD 0 vertices with each LOD's own triangles from the `.vtx` (each mesh's vertex numbers count from the
+same start in every LOD; checked against the six LODs of a Garry's Mod vehicle). Animations are decoded from the `.mdl`,
+or the `.ani` file beside it when they are in animation blocks, section by section: each animated bone has a raw
+rotation (Quaternion64 or Quaternion48), run-length encoded rotation tracks, a raw position (three half floats) or
+position tracks, scaled by the bone's scales and added to its bind pose (not for delta animations); bones an animation
+doesn't touch keep their bind pose, and the linear motion studiomdl took out of the root bones (movements) is added back.
+Rotations are Source's Euler angles (rotation about X, then Y, then Z), as in the bones and SMDs. The `.phy`'s solids
+are walked through their IVP ledge trees; points are in metres in IVP's axes (Source x = x, y = z, z = −y), turned into
+inches. A solid named after a bone (`$collisionjoints`, a ragdoll) is in that bone's space and is moved into the model's
+with the bind pose; the joint limits (`ragdollconstraint`) become `$jointconstrain` lines. A `.phy` with another
+checksum than the `.mdl` is left out with a comment in the QC. Not written: flexes (`.vta`), eyeballs, IK rules and
+chains, bone controllers and procedural bones; a model that has them still compiles from the QC, without them.
+Compiling a QC back into an MDL needs Valve's studiomdl, which only the games and their SDKs ship.
+
+Every count in the `.mdl`, `.phy` and `.ani` is capped and every offset checked against the file before it is used or
+anything is allocated: frames (100,000 per animation), bone entries per frame, track runs, sequences and their blend
+grids (4,096 animations), events, LODs (8), attachments, hitboxes, solids (1,024), ledge tree nodes (65,536, walked
+without recursion so a looping tree stops), points per piece (65,536); a damaged file is refused and the half-written
+folder removed.
+
**3D → pictures**: a small software rasteriser (`MeshRenderer`) draws an isometric view from the front right, above,
with one light and flat shading per triangle, vertex colours or the texture interpolated across it, and a z-buffer,
at 2048 px and scaled down to 1024 px for smooth edges, on a transparent background. The preset's image options
@@ -1824,6 +1887,9 @@ bundled with the installer), never a 7-Zip installed on the PC. It runs as a sep
| Split archives (.001, .7z.001, .zip.001, .tar.001) | 7-Zip (combining .002, .003 … automatically) | — |
| ALZ (ALZip) | Filee: store, deflate and ALZip's bzip2 variant | — |
| EGG (ALZip) | Filee: store, deflate, bzip2, LZMA (decoded by 7-Zip); several blocks per file | — |
+| GMA (Garry's Mod addon) | Filee: versions 1–3, CRCs checked, an `addon.json` written beside the files | Filee, as gmad does (see below) |
+| BSP (Source engine map), its packed files | Filee copies the pakfile lump (a ZIP) out, 7-Zip unpacks it | — |
+| VPK (Valve pak) | Filee: versions 1 and 2, `_dir.vpk` with its `_000.vpk` … parts, CRCs checked; a PS Vita `.vpk` (a ZIP) by 7-Zip | — |
- **Extract** (target `folder`): the archive is unpacked into a hidden folder next to the result and moved into place
when complete. When the archive holds exactly one top folder, its content becomes the result folder (`photos.zip`
@@ -1837,6 +1903,38 @@ bundled with the installer), never a 7-Zip installed on the PC. It runs as a sep
an archive or a folder whose only top folder holds the marker gets that folder's content, not the folder
(`GamePacks.Root`); an add-on (MCADDON, several packs side by side) is packed as it is. The presets (MCPACK …
MRPACK) are in the palette only.
+- **Garry's Mod addons** (`.gma`, #1526, the Games field; `Archives/Gma.cs`, Filee's own reader): the Steam
+ Workshop's format, a header (title, a JSON description with the type and tags, author, timestamp), a file list
+ (names, sizes, CRC-32s) and the files one after another, ended by a CRC-32 of everything before it. Every name is
+ checked before anything is written (an absolute name, a drive or a `..` stops the addon), the entries are capped at
+ the archive limit and their sizes must fit inside the addon. Unpacking checks every file's CRC (0, as `gmad -nocrc`
+ writes it, is not checked) and writes an `addon.json` with the title, description, type and tags, so the folder packs
+ back into the same addon; the required-content list and the author are not kept (Garry's Mod uses neither). Packing
+ a folder, a ZIP of one or the files dropped onto "Compress into one archive" reads `addon.json` like gmad: `title`
+ (else the folder's name), `type` (else `servercontent`), `tags`, `description` and the `ignore` wildcards. Only
+ files on gmad's whitelist go in (`lua/*.lua`, `materials/*.vtf`, `models/*.mdl` …, the ones Garry's Mod mounts;
+ `addon.json`, `thumbs.db`, `desktop.ini`, `.git*` and `.DS_Store` never), so a `.psd` or a `readme.txt` beside them
+ is left out rather than failing the addon as gmad does without `-warninvalid`; a folder with none of them is
+ refused. Names are stored in lower case, sorted, with a CRC each. The type and tags are written as given (lower
+ case), not checked against the Workshop's lists. The "GMA" preset is in the palette after the game packs (schema
+ 96), not on a donut.
+- **Valve paks** (`.vpk`, #1526, the Games field; `Archives/Vpk.cs`): the archives Source engine games keep their files
+ in (`hl2_textures_dir.vpk`, the content Garry's Mod mounts). The directory file holds a tree of extensions, folders
+ and names; each file's content is a few preload bytes stored in the tree plus its data, which follows the tree in the
+ directory file (archive index 0x7FFF) or sits in `_nnn.vpk` next to it. The directory file is what unpacks;
+ a data part on its own is refused with the name of its directory file, a missing part names the part. Every name is
+ checked before anything is written, the tree is capped at 256 MB and the entries at the archive limit, preload bytes
+ must lie inside the tree, parts are `_000` to `_999`, and each file's data must lie inside its part; the total is
+ checked against the free space first. The MD5 and signature sections of version 2 are not read; every file's CRC-32
+ is checked. Nothing writes VPK (Garry's Mod
+ loads addons as folders or GMA). A PS Vita package has the same extension but is a ZIP: it goes to 7-Zip.
+- **Source engine maps** (`.bsp`, #1526, the Games field; `Archives/Bsp.cs`): the custom textures, models and sounds
+ packed into a map (by VBSP, VIDE or bspzip) are a ZIP stored as lump 40, the pakfile. Filee finds it in the lump
+ directory (offset, length, version, code; Left 4 Dead 2's version, offset, length order is used when only that one
+ points inside the file), copies it to the work folder and 7-Zip unpacks it, so a map extracts or converts to any
+ archive like one, with the same checks as any ZIP (names leaving the folder, the entry limit, the free space). The
+ lump must lie inside the map, and its copy is bounded by the lump's length. A map without packed files (an empty ZIP), a compressed pakfile lump and a Quake or GoldSrc map
+ (no `VBSP`) are refused with a message saying so. The map's geometry is not read.
- **Adobe extension packages** (`.zxp`, #1382, read only like JAR): the panels and plug-ins of After Effects, Premiere
Pro, Photoshop and the other Creative Cloud apps are signed ZIPs (`mimetype`, the extension's files and
`META-INF/signatures.xml`). They extract and convert to every archive target with the signature file inside.
diff --git a/src/Filee.Core/Formats/FormatFields.cs b/src/Filee.Core/Formats/FormatFields.cs
index 500ed61c3..a09650158 100644
--- a/src/Filee.Core/Formats/FormatFields.cs
+++ b/src/Filee.Core/Formats/FormatFields.cs
@@ -83,7 +83,7 @@ .. Enum.GetValues().Select(category => category.ToString().ToLow
"dts", "tar", "tgz", "tbz2", "txz", "tar7z", "tlz", "gz", "bz2", "xz", "lz", "lzma", "jar", "zxp", "nupkg", "apks",
"aab", "mcpack", "mcaddon", "mcworld", "mctemplate", "mrpack", "cab", "gerber",
"cpio", "deb", "rpm", "dwg", "ttf", "otf", "woff", "woff2", "ass", "ssa", "ttml", "scc", "glb", "gltf",
- "stl", "obj", "3mf", "ply", "fbx", "gcode", "geojson", "kml", "kmz", "gpx", "shp", "fit", "osm", "xm", "it", "ani", "vtf"
+ "stl", "obj", "3mf", "ply", "fbx", "gcode", "geojson", "kml", "kmz", "gpx", "shp", "fit", "osm", "xm", "it", "ani", "vtf", "gma", "vpk", "bsp", "mdl", "smd"
]),
(FormatGrade.Legacy,
[
@@ -100,7 +100,7 @@ .. Enum.GetValues().Select(category => category.ToString().ToLow
// Layout, print, photo post-production, signs and embroidery.
(["psd", "psb", "exr", "hdr", "xcf", "kra", "pdn", "raw", "eps", "ps", "ai", "cdr", "plt", "dst"], ["design"]),
// Game textures and cutscenes, Minecraft packs and modpacks.
- (["dds", "vtf", "usm", "bik", "mcpack", "mcaddon", "mcworld", "mctemplate", "mrpack"], ["game"]),
+ (["dds", "vtf", "usm", "bik", "mcpack", "mcaddon", "mcworld", "mctemplate", "mrpack", "gma", "vpk", "bsp", "mdl", "smd"], ["game"]),
// Medical imaging and astronomy.
(["dcm", "fits"], ["science"]),
// Software work: raw image formats of graphics tools, app icons and cursors, config and data files, flat XML
diff --git a/src/Filee.Core/Formats/FormatRegistry.cs b/src/Filee.Core/Formats/FormatRegistry.cs
index 86df633b1..f3bb049a6 100644
--- a/src/Filee.Core/Formats/FormatRegistry.cs
+++ b/src/Filee.Core/Formats/FormatRegistry.cs
@@ -355,6 +355,14 @@ public static class FormatRegistry
new("mcworld", "Minecraft world", FormatCategory.Archive, ["mcworld"]),
new("mctemplate", "Minecraft world template", FormatCategory.Archive, ["mctemplate"]),
new("mrpack", "Modrinth modpack", FormatCategory.Archive, ["mrpack"]),
+ // Garry's Mod addons (Steam Workshop), read and written by Filee as gmad does: addon.json gives the title, type,
+ // tags and ignores, and only the files Garry's Mod loads go in.
+ new("gma", "Garry's Mod addon (GMA)", FormatCategory.Archive, ["gma"]),
+ // Valve paks (Source engine games' files, "pak01_dir.vpk" with its "pak01_000.vpk" … parts), read by Filee; a PS
+ // Vita package under the same extension is a ZIP and goes to 7-Zip.
+ new("vpk", "Valve pak (VPK)", FormatCategory.Archive, ["vpk"], Writable: false),
+ // Source engine maps: the files packed into a map (its pakfile lump, a ZIP) unpack like an archive.
+ new("bsp", "Source map (BSP)", FormatCategory.Archive, ["bsp"], Writable: false),
new("cab", "CAB", FormatCategory.Archive, ["cab"], Writable: false),
new("cpio", "CPIO", FormatCategory.Archive, ["cpio"], Writable: false),
new("deb", "DEB", FormatCategory.Archive, ["deb"], Writable: false),
@@ -409,6 +417,11 @@ public static class FormatRegistry
new("3mf", "3MF", FormatCategory.Model3D, ["3mf"]),
new("ply", "PLY", FormatCategory.Model3D, ["ply"]),
new("fbx", "FBX", FormatCategory.Model3D, ["fbx"], Writable: false),
+ // Source engine models (Garry's Mod and other Source games), read with the .vvd and .vtx beside them and their
+ // textures from the "materials" folder above them.
+ new("mdl", "Source model (MDL)", FormatCategory.Model3D, ["mdl"], Writable: false),
+ // Studiomdl data, the text the Source engine's model compiler reads (from an MDL with its bones and weights).
+ new("smd", "Studiomdl data (SMD)", FormatCategory.Model3D, ["smd"]),
// Geospatial vector data (CSV with coordinates is a Data format and reaches these through the geo engine)
new("geojson", "GeoJSON", FormatCategory.Geo, ["geojson"]),
diff --git a/src/Filee.Core/Presets/BuiltInData.cs b/src/Filee.Core/Presets/BuiltInData.cs
index 7fe755d92..d888d97a0 100644
--- a/src/Filee.Core/Presets/BuiltInData.cs
+++ b/src/Filee.Core/Presets/BuiltInData.cs
@@ -160,6 +160,7 @@ .. CatalogPresets(),
.. DataPresets(),
.. CodecAndSubtitlePresets(),
.. ModelPresets(),
+ .. SourceModelPresets(),
.. GeoPresets(),
.. TracePresets(),
];
@@ -226,8 +227,8 @@ public static List ClipPresets() =>
];
///
- /// Minecraft packs and Modrinth modpacks (#824, #898, added to the library in settings schema 62): in the palette
- /// only, after "TAR", in the Games field (off in a new install).
+ /// Minecraft packs and Modrinth modpacks (#824, #898, added to the library in settings schema 62) and Garry's Mod
+ /// addons (#1526, schema 96): in the palette only, after "TAR", in the Games field (off in a new install).
///
public static List GamePackPresets() =>
[
@@ -236,6 +237,7 @@ public static List GamePackPresets() =>
new() { Id = "to-mcworld", Name = "MCWORLD", TargetFormat = "mcworld" },
new() { Id = "to-mctemplate", Name = "MCTEMPLATE", TargetFormat = "mctemplate" },
new() { Id = "to-mrpack", Name = "MRPACK", TargetFormat = "mrpack" },
+ new() { Id = "to-gma", Name = "GMA", TargetFormat = "gma" },
];
///
@@ -590,6 +592,20 @@ public static List ModelPresets() =>
new() { Id = "to-gltf", Name = "glTF", TargetFormat = "gltf" },
];
+ ///
+ /// Studiomdl data for Source engine models (#1526, added to the library in settings schema 97): in the palette only,
+ /// after glTF, in the Games field (off in a new install). "QC + SMD" decompiles a model into the folder studiomdl
+ /// builds it from (settings schema 98); only a .mdl can take it.
+ ///
+ public static List SourceModelPresets() =>
+ [
+ new() { Id = "to-smd", Name = "SMD", TargetFormat = "smd" },
+ new() { Id = DecompileModelId, Name = "QC + SMD", TargetFormat = FormatRegistry.Folder },
+ ];
+
+ /// The preset that decompiles a Source engine model (.mdl) into its QC and SMDs.
+ public const string DecompileModelId = "decompile-mdl";
+
///
/// Geospatial vector targets (added in settings schema 25, see SettingsMigrations). CSV, the sixth slice of the
/// donut, is the spreadsheet preset. KMZ is a KML in a ZIP and is on no donut.
diff --git a/src/Filee.Core/Settings/AppSettings.cs b/src/Filee.Core/Settings/AppSettings.cs
index da022ae84..0c9a4ad5d 100644
--- a/src/Filee.Core/Settings/AppSettings.cs
+++ b/src/Filee.Core/Settings/AppSettings.cs
@@ -170,7 +170,7 @@ public static bool IsFlipKeyAllowed(string key, IEnumerable gest
public sealed class AppSettings
{
/// Current schema version. Increase when the format changes and add a migration.
- public const int CurrentSchemaVersion = 94;
+ public const int CurrentSchemaVersion = 98;
public int SchemaVersion { get; set; } = CurrentSchemaVersion;
diff --git a/src/Filee.Core/Settings/UserDataStore.cs b/src/Filee.Core/Settings/UserDataStore.cs
index f72e1b754..1a1523d59 100644
--- a/src/Filee.Core/Settings/UserDataStore.cs
+++ b/src/Filee.Core/Settings/UserDataStore.cs
@@ -1176,6 +1176,27 @@ public static bool ApplyToLibrary(int from, List presets, List p.Id != BuiltInData.DecompileModelId))
+ changed |= AddLikeNewLibrary(presets, BuiltInData.DecompileModelId);
return changed;
}
@@ -1233,6 +1254,37 @@ private static bool AddGamePacks(List presets)
return changed;
}
+ ///
+ /// "GMA" after the last game pack of the library (else after "TAR", else at the end); the game packs the user removed
+ /// stay removed.
+ ///
+ private static bool AddGma(List presets)
+ {
+ if (presets.Any(p => p.Id == "to-gma"))
+ return false;
+ var packs = BuiltInData.GamePackPresets().Select(p => p.Id).ToHashSet();
+ var at = presets.FindLastIndex(p => packs.Contains(p.Id));
+ if (at < 0)
+ at = presets.FindIndex(p => p.Id == "to-tar");
+ presets.Insert(at < 0 ? presets.Count : at + 1, BuiltInData.GamePackPresets().Single(p => p.Id == "to-gma"));
+ return true;
+ }
+
+ /// "SMD" after the last 3D model preset of the library (glTF), or at the end.
+ private static bool AddSourcePresets(List presets)
+ {
+ var changed = false;
+ // Only "SMD": "QC + SMD" came a step later.
+ foreach (var preset in BuiltInData.SourceModelPresets().Where(p => p.Id == "to-smd" && presets.All(existing => existing.Id != p.Id)))
+ {
+ var models = BuiltInData.ModelPresets().Select(p => p.Id).ToHashSet();
+ var last = presets.FindLastIndex(p => models.Contains(p.Id));
+ presets.Insert(last < 0 ? presets.Count : last + 1, preset);
+ changed = true;
+ }
+ return changed;
+ }
+
private static bool AddCropVideo(List presets)
{
var at = presets.FindIndex(p => p.Id == "tool-clip");
diff --git a/src/Filee.Engines/Archives/ArchiveConverter.cs b/src/Filee.Engines/Archives/ArchiveConverter.cs
index 1ec9258aa..c7bf17ed4 100644
--- a/src/Filee.Engines/Archives/ArchiveConverter.cs
+++ b/src/Filee.Engines/Archives/ArchiveConverter.cs
@@ -1,6 +1,6 @@
// Archives: unpack (to a folder, the "Extract" preset), convert between archive formats (ZIP, 7Z, TAR, TAR.GZ,
-// TAR.BZ2, TAR.XZ, single-file GZ / BZ2 / XZ, and the Minecraft and Modrinth packs that are ZIPs, see GamePacks), and
-// "Compress into one archive" for any files (IFileCombiner).
+// TAR.BZ2, TAR.XZ, single-file GZ / BZ2 / XZ, the Minecraft and Modrinth packs that are ZIPs, see GamePacks, and Garry's
+// Mod addons, see Gma), and "Compress into one archive" for any files (IFileCombiner).
// Reading uses the bundled 7-Zip (engines/7zip, LGPL, separate process) plus Filee's own ALZ, EGG and lzip readers;
// ZIP, TAR and TAR.GZ are written in-process, the other formats by 7-Zip. See docs/ENGINES.md.
@@ -16,7 +16,7 @@ namespace Filee.Engines.Archives;
public sealed class ArchiveConverter : IConverter, IFileCombiner
{
/// Archive formats Filee writes (plus ).
- private static readonly string[] ArchiveTargets = ["zip", "7z", "tar", "tgz", "tbz2", "txz", .. GamePacks.Formats];
+ private static readonly string[] ArchiveTargets = ["zip", "7z", "tar", "tgz", "tbz2", "txz", "gma", .. GamePacks.Formats];
/// Compressed single files (no names or folders inside); they convert among each other.
private static readonly string[] SingleFileSources = ["gz", "bz2", "xz", "lzma", "z", "lz"];
@@ -38,7 +38,7 @@ .. from source in SingleFileSources
from target in ArchivePacker.SingleFileFormats
select new ConversionEdge(source, target),
.. ArchiveExtractor.ReadableFormats.Select(source => new ConversionEdge(source, "iso", DirectOnly: true)),
- .. new[] { "zip", "7z", "tar", "tgz", "tbz2", "txz" }
+ .. new[] { "zip", "7z", "tar", "tgz", "tbz2", "txz", "gma" }
.Select(target => new ConversionEdge(FormatRegistry.Folder, target, DirectOnly: true)),
];
@@ -85,6 +85,8 @@ public async Task> ConvertAsync(ConversionStep step, IProg
}
if (GamePacks.Formats.Contains(step.To))
unpacked = GamePacks.Root(unpacked, step.To);
+ else if (step.To == "gma")
+ unpacked = Gma.Root(unpacked);
var items = Directory.EnumerateFileSystemEntries(unpacked)
.Order(StringComparer.Ordinal)
.Select(path => new PackItem(path, Path.GetFileName(path)))
@@ -141,10 +143,11 @@ public async Task CombineAsync(IReadOnlyList inputPaths, string outputPa
string workDirectory, IProgress? progress, CancellationToken cancellationToken)
{
var sevenZip = ArchivePacker.BuiltInFormats.Contains(format) ? null : _sevenZip ??= SevenZip.Locate();
- // A pack made from its folder holds the folder's content, so its marker file is at the top.
- var entries = GamePacks.Formats.Contains(format) && inputPaths is [var folder] && Directory.Exists(folder)
- ? Directory.EnumerateFileSystemEntries(GamePacks.Root(folder, format)).Order(StringComparer.Ordinal)
- .Select(path => new PackItem(path, Path.GetFileName(path))).ToList()
+ // A pack (or a Garry's Mod addon) made from its folder holds the folder's content, so its marker file (addon.json)
+ // is at the top.
+ var entries = (GamePacks.Formats.Contains(format) || format == "gma") && inputPaths is [var folder] && Directory.Exists(folder)
+ ? Directory.EnumerateFileSystemEntries(format == "gma" ? Gma.Root(folder) : GamePacks.Root(folder, format))
+ .Order(StringComparer.Ordinal).Select(path => new PackItem(path, Path.GetFileName(path))).ToList()
: UniqueEntries(inputPaths);
await new ArchivePacker(sevenZip).PackAsync(entries, outputPath, format, preset.Archive.Level,
workDirectory, progress, cancellationToken);
diff --git a/src/Filee.Engines/Archives/ArchiveExtractor.cs b/src/Filee.Engines/Archives/ArchiveExtractor.cs
index 66a57e109..405245f65 100644
--- a/src/Filee.Engines/Archives/ArchiveExtractor.cs
+++ b/src/Filee.Engines/Archives/ArchiveExtractor.cs
@@ -1,6 +1,8 @@
// Unpacks every archive format Filee reads into a folder: the bundled 7-Zip for most of them, Filee's own readers for
-// ALZ, EGG and lzip. Compressed TARs (.tar.gz, .tar.lz …) and RPM payloads are unpacked stage by stage until the
-// files appear. Every stage is checked first (encryption, paths leaving the folder, disk space) and links are removed.
+// ALZ, EGG, lzip, Garry's Mod addons (GMA) and Valve paks (VPK); the files packed into a Source engine map (BSP) are a
+// ZIP that Filee copies out for 7-Zip. Compressed TARs (.tar.gz, .tar.lz …) and RPM payloads are unpacked stage by
+// stage until the files appear. Every stage is checked first (encryption, paths leaving the folder, disk space) and
+// links are removed.
// A single compressed stream (.gz, .bz2 …) is written by Filee as 7-Zip decodes it, so it can't fill the drive. The
// archive's Mark of the Web goes onto the unpacked files.
@@ -25,7 +27,7 @@ internal sealed class ArchiveExtractor(SevenZip? sevenZip, Func? s
];
/// Formats read by Filee's own code (lzip still needs 7-Zip's LZMA decoder).
- public static readonly IReadOnlyList BuiltInFormats = ["alz", "egg", "lz", "tlz"];
+ public static readonly IReadOnlyList BuiltInFormats = ["alz", "egg", "lz", "tlz", "gma", "vpk", "bsp"];
/// Everything that can be unpacked.
public static IEnumerable ReadableFormats => SevenZipFormats.Concat(BuiltInFormats);
@@ -85,6 +87,34 @@ private async Task ExtractOnceAsync(string archive, string format, string folder
await EggReader.ExtractAsync(archive, folder, sevenZip, scratchRoot, cancellationToken);
PassOnMark(archive, folder);
break;
+ case "gma":
+ await Task.Run(() => Gma.Extract(archive, folder, cancellationToken), cancellationToken);
+ PassOnMark(archive, folder);
+ break;
+ // A PS Vita package is a ZIP under the same extension.
+ case "vpk" when Vpk.IsZip(archive):
+ await ExtractOnceAsync(archive, "zip", folder, scratchRoot, cancellationToken);
+ break;
+ case "vpk":
+ await Task.Run(() => Vpk.Extract(archive, folder, cancellationToken), cancellationToken);
+ PassOnMark(archive, folder);
+ break;
+ case "bsp":
+ {
+ // The map's packed files are a ZIP inside it: copied out, then unpacked like any ZIP.
+ var pakfile = Path.Combine(scratchRoot, $"pakfile-{Guid.NewGuid():N}.zip");
+ try
+ {
+ await Task.Run(() => Bsp.ExtractPakfile(archive, pakfile, cancellationToken), cancellationToken);
+ await ExtractOnceAsync(pakfile, "zip", folder, scratchRoot, cancellationToken);
+ }
+ finally
+ {
+ ArchiveFiles.TryDeleteFile(pakfile);
+ }
+ PassOnMark(archive, folder);
+ break;
+ }
case "lz":
case "tlz":
var output = FormatRegistry.NameWithoutExtension(archive) + (format == "tlz" ? ".tar" : "");
diff --git a/src/Filee.Engines/Archives/ArchivePacker.cs b/src/Filee.Engines/Archives/ArchivePacker.cs
index 442d1fe26..8172a99ae 100644
--- a/src/Filee.Engines/Archives/ArchivePacker.cs
+++ b/src/Filee.Engines/Archives/ArchivePacker.cs
@@ -1,10 +1,12 @@
-// Writes archives. ZIP (and the game packs that are ZIPs, see GamePacks), TAR and TAR.GZ are written in-process (System.IO.Compression / System.Formats.Tar), so file
+// Writes archives. ZIP (and the game packs that are ZIPs, see GamePacks), TAR, TAR.GZ and Garry's Mod addons (GMA, see Gma)
+// are written in-process (System.IO.Compression / System.Formats.Tar / Filee's own writer), so file
// names are exactly what Filee chooses: UTF-8 in ZIP (with the language encoding flag), PAX headers in TAR. 7Z, the
// bzip2 / xz TAR variants and single-file GZ / BZ2 / XZ are written by the bundled 7-Zip.
using System.Formats.Tar;
using System.IO.Compression;
using System.Text;
+using Filee.Core.Formats;
using Filee.Core.Presets;
using Filee.Engines.Infrastructure;
@@ -18,7 +20,7 @@ internal sealed class ArchivePacker(SevenZip? sevenZip)
{
/// Formats written without 7-Zip.
public static readonly IReadOnlySet BuiltInFormats =
- new HashSet(["zip", "tar", "tgz", .. GamePacks.Formats], StringComparer.OrdinalIgnoreCase);
+ new HashSet(["zip", "tar", "tgz", "gma", .. GamePacks.Formats], StringComparer.OrdinalIgnoreCase);
/// Formats that hold exactly one file, without name or folders.
public static readonly IReadOnlySet SingleFileFormats = new HashSet(StringComparer.OrdinalIgnoreCase) { "gz", "bz2", "xz" };
@@ -41,6 +43,12 @@ public async Task PackAsync(IReadOnlyList items, string outputPath, st
case "zip" or "mcpack" or "mcaddon" or "mcworld" or "mctemplate" or "mrpack":
await Task.Run(() => WriteZip(items, temp, level, progress, cancellationToken), cancellationToken);
break;
+ case "gma":
+ // The addon's title, without an addon.json, is the name it is saved under (its folder's name).
+ await Task.Run(() => Gma.Write(
+ [.. Expand(items).Where(f => !f.IsFolder).Select(f => (f.EntryName, f.Path))], temp,
+ FormatRegistry.NameWithoutExtension(outputPath), progress, cancellationToken), cancellationToken);
+ break;
case "tar":
case "tgz":
await Task.Run(() => WriteTar(items, temp, format == "tgz" ? level : null, progress, cancellationToken), cancellationToken);
diff --git a/src/Filee.Engines/Archives/Bsp.cs b/src/Filee.Engines/Archives/Bsp.cs
new file mode 100644
index 000000000..141aa8750
--- /dev/null
+++ b/src/Filee.Engines/Archives/Bsp.cs
@@ -0,0 +1,65 @@
+// Source engine maps (.bsp): the files packed into a map (custom textures, models and sounds; the "pakfile" lump, made by
+// VBSP and tools such as VIDE or bspzip) are a ZIP stored inside the map. Filee copies that ZIP out and 7-Zip unpacks it.
+//
+// "VBSP" + version i32 (19/20 in Garry's Mod, up to 21), then 64 lumps of 16 bytes and the map revision;
+// a lump is offset i32, length i32, version i32 and a four-byte code (its uncompressed size when it is compressed).
+// Left 4 Dead 2 writes version, offset, length instead; the order that points inside the file is used.
+// Lump 40 is the pakfile.
+
+using System.Buffers.Binary;
+
+namespace Filee.Engines.Archives;
+
+/// Takes the packed files out of Source engine maps.
+internal static class Bsp
+{
+ private const int LumpCount = 64;
+ private const int PakfileLump = 40;
+
+ /// "VBSP", the version and the lump directory (the map revision follows).
+ private const int HeaderSize = 8 + LumpCount * 16 + 4;
+
+ /// An empty ZIP: just its end record.
+ private const int EmptyZip = 22;
+
+ /// Where the pakfile lump of is.
+ internal static (long Offset, long Length) Pakfile(Stream map, string mapName)
+ {
+ Span header = stackalloc byte[HeaderSize];
+ if (map.ReadAtLeast(header, HeaderSize, throwOnEndOfStream: false) < HeaderSize || !header.StartsWith("VBSP"u8))
+ throw new InvalidDataException($"{mapName} is not a Source engine map (VBSP).");
+ var version = BinaryPrimitives.ReadInt32LittleEndian(header[4..]);
+ var lump = header.Slice(8 + PakfileLump * 16, 16);
+ long a = BinaryPrimitives.ReadInt32LittleEndian(lump);
+ long b = BinaryPrimitives.ReadInt32LittleEndian(lump[4..]);
+ long c = BinaryPrimitives.ReadInt32LittleEndian(lump[8..]);
+ var compressed = BinaryPrimitives.ReadInt32LittleEndian(lump[12..]) != 0;
+
+ bool Inside(long offset, long length) => length == 0 || offset >= HeaderSize && length > 0 && offset + length <= map.Length;
+ (long Offset, long Length) found;
+ if (Inside(a, b))
+ found = (a, b);
+ else if (version >= 21 && Inside(b, c)) // Left 4 Dead 2: version, offset, length
+ found = (b, c);
+ else
+ throw ArchiveSafety.Damaged(mapName, "the packed files point outside the map");
+
+ if (found.Length <= EmptyZip)
+ throw new InvalidDataException($"{mapName} has no files packed into the map; its textures and models come from the game or an addon.");
+ if (compressed)
+ throw new InvalidDataException($"{mapName} keeps its packed files compressed (LZMA lump), which Filee does not unpack.");
+ return found;
+ }
+
+ /// Copies the packed files' ZIP of to .
+ public static void ExtractPakfile(string map, string zipPath, CancellationToken cancellationToken)
+ {
+ var mapName = Path.GetFileName(map);
+ using var stream = new FileStream(map, FileMode.Open, FileAccess.Read, FileShare.Read, 1 << 16);
+ var (offset, length) = Pakfile(stream, mapName);
+ stream.Position = offset;
+ using var output = new FileStream(zipPath, FileMode.CreateNew, FileAccess.Write, FileShare.None, 1 << 16);
+ if (ArchiveFiles.CopyLimited(new BoundedStream(stream, length), output, length, cancellationToken) != length)
+ throw ArchiveSafety.Damaged(mapName, "the packed files are cut off");
+ }
+}
diff --git a/src/Filee.Engines/Archives/Gma.cs b/src/Filee.Engines/Archives/Gma.cs
new file mode 100644
index 000000000..bbc9e839d
--- /dev/null
+++ b/src/Filee.Engines/Archives/Gma.cs
@@ -0,0 +1,450 @@
+// Garry's Mod addons (.gma), the format of the Steam Workshop: read and written by Filee, as Facepunch's gmad does.
+// Layout (little endian, from gmad's AddonFormat.h, AddonReader.h and create_gmad.cpp):
+//
+// "GMAD" + version (1 byte, 1–3; gmad writes 3)
+// SteamID u64 (unused, 0) + timestamp u64 (Unix seconds)
+// version > 1: required content, NUL-terminated strings ending with an empty one
+// title, description, author (NUL-terminated); addon version i32 (unused, 1)
+// file list: per file a number u32 (1, 2, …), its name (NUL-terminated, lower case), size i64 and CRC-32 u32;
+// a number 0 ends the list
+// the files' content one after another, then a CRC-32 of everything before it (0 when gmad ran with -nocrc)
+//
+// The description is the JSON gmad builds from addon.json ({"description", "type", "tags"}). Unpacking writes an
+// addon.json beside the files, so the folder packs back into the same addon. Packing reads it like gmad: title, type,
+// tags and the "ignore" wildcards; files outside gmad's whitelist (the only ones Garry's Mod loads) are left out, and
+// names are stored in lower case.
+
+using System.Text;
+using System.Text.Json;
+using System.Text.Json.Nodes;
+
+namespace Filee.Engines.Archives;
+
+/// Unpacks and writes Garry's Mod addons.
+internal static class Gma
+{
+ private static ReadOnlySpan Ident => "GMAD"u8;
+
+ /// The version gmad writes.
+ private const byte Version = 3;
+
+ /// The settings file of an addon folder; never packed, written back on unpacking.
+ public const string AddonJson = "addon.json";
+
+ ///
+ /// gmad's whitelist (AddonWhiteList.h), in order: the files Garry's Mod mounts from an addon. A "!" entry takes back
+ /// what the entries before it allowed. "*" matches any characters, slashes included.
+ ///
+ private static readonly string[] WhiteList =
+ [
+ "lua/*.lua", "scenes/*.vcd", "particles/*.pcf", "resource/fonts/*.ttf", "scripts/vehicles/*.txt",
+ "resource/localization/*/*.properties", "maps/*.bsp", "maps/*.lmp", "maps/*.nav", "maps/*.ain", "maps/thumb/*.png",
+ "sound/*.wav", "sound/*.mp3", "sound/*.ogg", "materials/*.vmt", "materials/*.vtf", "materials/*.png",
+ "materials/*.jpg", "materials/*.jpeg", "materials/colorcorrection/*.raw", "models/*.mdl", "models/*.phy",
+ "models/*.ani", "models/*.vvd", "models/*.vtx", "!models/*.sw.vtx", "!models/*.360.vtx", "!models/*.xbox.vtx",
+ "gamemodes/*/*.txt", "!gamemodes/*/*/*.txt", "gamemodes/*/*.fgd", "!gamemodes/*/*/*.fgd", "gamemodes/*/logo.png",
+ "gamemodes/*/icon24.png", "gamemodes/*/gamemode/*.lua", "gamemodes/*/entities/effects/*.lua",
+ "gamemodes/*/entities/weapons/*.lua", "gamemodes/*/entities/entities/*.lua", "gamemodes/*/backgrounds/*.png",
+ "gamemodes/*/backgrounds/*.jpg", "gamemodes/*/backgrounds/*.jpeg", "gamemodes/*/content/models/*.mdl",
+ "gamemodes/*/content/models/*.phy", "gamemodes/*/content/models/*.ani", "gamemodes/*/content/models/*.vvd",
+ "gamemodes/*/content/models/*.vtx", "!gamemodes/*/content/models/*.sw.vtx", "!gamemodes/*/content/models/*.360.vtx",
+ "!gamemodes/*/content/models/*.xbox.vtx", "gamemodes/*/content/materials/*.vmt", "gamemodes/*/content/materials/*.vtf",
+ "gamemodes/*/content/materials/*.png", "gamemodes/*/content/materials/*.jpg", "gamemodes/*/content/materials/*.jpeg",
+ "gamemodes/*/content/materials/colorcorrection/*.raw", "gamemodes/*/content/scenes/*.vcd",
+ "gamemodes/*/content/particles/*.pcf", "gamemodes/*/content/resource/fonts/*.ttf",
+ "gamemodes/*/content/scripts/vehicles/*.txt", "gamemodes/*/content/resource/localization/*/*.properties",
+ "gamemodes/*/content/maps/*.bsp", "gamemodes/*/content/maps/*.nav", "gamemodes/*/content/maps/*.ain",
+ "gamemodes/*/content/maps/thumb/*.png", "gamemodes/*/content/sound/*.wav", "gamemodes/*/content/sound/*.mp3",
+ "gamemodes/*/content/sound/*.ogg", "data_static/*.txt", "data_static/*.dat", "data_static/*.json",
+ "data_static/*.xml", "data_static/*.csv", "shaders/fxc/*.vcs",
+ ];
+
+ /// The longest name Filee reads from a file list (gmad reserves 180 bytes; longer is a damaged file).
+ private const int MaxNameBytes = 4096;
+
+ /// One file of an addon.
+ internal sealed record Entry(string Name, long Size, uint Crc, long Offset);
+
+ /// What the header says about an addon.
+ internal sealed record Header(string Title, string Description, string? Type, IReadOnlyList Tags, string Author,
+ DateTimeOffset? Created, IReadOnlyList Entries);
+
+ /// True for a file that starts like an addon.
+ public static bool IsGma(ReadOnlySpan start) => start.StartsWith(Ident);
+
+ ///
+ /// Unpacks every file of into , with an addon.json for
+ /// its title, description, type and tags.
+ ///
+ public static void Extract(string archive, string targetDirectory, CancellationToken cancellationToken)
+ {
+ var archiveName = Path.GetFileName(archive);
+ using var stream = new FileStream(archive, FileMode.Open, FileAccess.Read, FileShare.Read, 1 << 16);
+ var header = ReadHeader(stream, archiveName);
+ // Every name is checked before anything is written: one that leaves the folder stops the whole addon.
+ foreach (var entry in header.Entries)
+ ArchiveSafety.ResolveEntryPath(targetDirectory, entry.Name, archiveName);
+ ArchiveSafety.EnsureRoom(header.Entries.Sum(e => e.Size), targetDirectory, archiveName);
+
+ var dataStart = stream.Position;
+ foreach (var entry in header.Entries)
+ {
+ cancellationToken.ThrowIfCancellationRequested();
+ var path = ArchiveSafety.ResolveEntryPath(targetDirectory, entry.Name, archiveName);
+ Directory.CreateDirectory(Path.GetDirectoryName(path)!);
+ path = ArchiveFiles.UniquePath(path);
+ stream.Position = dataStart + entry.Offset;
+ var crc = new Crc32();
+ using (var output = new CrcStream(new FileStream(path, FileMode.CreateNew, FileAccess.Write, FileShare.None, 1 << 16), crc))
+ ArchiveFiles.CopyLimited(new BoundedStream(stream, entry.Size), output, entry.Size, cancellationToken);
+ if (new FileInfo(path).Length != entry.Size)
+ throw ArchiveSafety.Damaged(archiveName, $"{entry.Name} is cut off");
+ // gmad -nocrc writes 0 for every file.
+ if (entry.Crc != 0 && crc.Value != entry.Crc)
+ throw ArchiveSafety.Damaged(archiveName, $"checksum error in {entry.Name}");
+ if (header.Created is { } created)
+ File.SetLastWriteTimeUtc(path, created.UtcDateTime);
+ }
+
+ var json = Path.Combine(targetDirectory, AddonJson);
+ if (!File.Exists(json))
+ File.WriteAllText(json, AddonJsonText(header), new UTF8Encoding(false));
+ }
+
+ /// Reads the header and file list; leaves at the first file's content.
+ internal static Header ReadHeader(Stream stream, string archiveName)
+ {
+ try
+ {
+ Span ident = stackalloc byte[4];
+ stream.ReadExactly(ident);
+ if (!ident.SequenceEqual(Ident))
+ throw new InvalidDataException($"{archiveName} is not a Garry's Mod addon (GMA).");
+ var reader = new BinaryReader(stream, Encoding.UTF8, leaveOpen: true);
+ var version = reader.ReadByte();
+ if (version is 0 or > Version)
+ throw new InvalidDataException($"{archiveName} is a GMA of version {version}; Filee reads versions 1 to {Version}.");
+ reader.ReadUInt64(); // SteamID
+ var timestamp = reader.ReadUInt64();
+ if (version > 1)
+ {
+ while (ReadString(stream, archiveName).Length > 0)
+ {
+ // required content, unused by Garry's Mod
+ }
+ }
+ var title = ReadString(stream, archiveName);
+ var description = ReadString(stream, archiveName);
+ var author = ReadString(stream, archiveName);
+ reader.ReadInt32(); // addon version
+
+ var entries = new List();
+ long offset = 0;
+ while (reader.ReadUInt32() != 0)
+ {
+ if (entries.Count >= ArchiveSafety.MaxEntries)
+ throw new InvalidDataException($"{archiveName} has more than {ArchiveSafety.MaxEntries:N0} files; Filee does not unpack it.");
+ var name = ReadString(stream, archiveName);
+ var size = reader.ReadInt64();
+ var crc = reader.ReadUInt32();
+ if (size < 0 || name.Length == 0)
+ throw ArchiveSafety.Damaged(archiveName, "the file list is broken");
+ // Checked per file, so the sum never overflows: no file may be larger than what is left of the addon.
+ if (size > stream.Length - stream.Position - offset)
+ throw ArchiveSafety.Damaged(archiveName, $"{name} is larger than the addon");
+ entries.Add(new Entry(name, size, crc, offset));
+ offset += size;
+ }
+ // Every file's content must be there (the CRC of the whole addon follows it).
+ if (stream.Position + offset > stream.Length)
+ throw ArchiveSafety.Damaged(archiveName, "the addon is cut off");
+
+ var (text, type, tags) = ParseDescription(description);
+ DateTimeOffset? created = timestamp is > 0 and < 253402300800 ? DateTimeOffset.FromUnixTimeSeconds((long)timestamp) : null;
+ return new Header(title, text, type, tags, author, created, entries);
+ }
+ catch (EndOfStreamException)
+ {
+ throw ArchiveSafety.Damaged(archiveName, "the addon is cut off");
+ }
+ }
+
+ /// A NUL-terminated UTF-8 string.
+ private static string ReadString(Stream stream, string archiveName)
+ {
+ var bytes = new List();
+ while (true)
+ {
+ var b = stream.ReadByte();
+ if (b < 0)
+ throw new EndOfStreamException();
+ if (b == 0)
+ return Encoding.UTF8.GetString([.. bytes]);
+ if (bytes.Count >= MaxNameBytes)
+ throw ArchiveSafety.Damaged(archiveName, "a name in the addon is too long");
+ bytes.Add((byte)b);
+ }
+ }
+
+ ///
+ /// The description, type and tags from gmad's JSON description; an old addon whose description is plain text keeps it
+ /// as it is.
+ ///
+ internal static (string Description, string? Type, IReadOnlyList Tags) ParseDescription(string description)
+ {
+ try
+ {
+ if (JsonNode.Parse(description) is JsonObject json)
+ {
+ var text = json["description"] is JsonValue d && d.TryGetValue(out var s) ? s : "";
+ var type = json["type"] is JsonValue t && t.TryGetValue(out var typeName) ? typeName : null;
+ var tags = json["tags"] is JsonArray array
+ ? array.OfType().Select(v => v.TryGetValue(out var tag) ? tag : null).OfType().ToList()
+ : [];
+ return (text, type, tags);
+ }
+ }
+ catch (JsonException)
+ {
+ // plain text
+ }
+ return (description, null, []);
+ }
+
+ /// The addon.json for an unpacked addon, as gmad reads it.
+ internal static string AddonJsonText(Header header)
+ {
+ var json = new JsonObject
+ {
+ ["title"] = header.Title,
+ ["type"] = header.Type ?? "",
+ ["tags"] = new JsonArray([.. header.Tags.Select(tag => (JsonNode?)JsonValue.Create(tag))]),
+ ["description"] = header.Description,
+ ["ignore"] = new JsonArray(),
+ };
+ return json.ToJsonString(new JsonSerializerOptions { WriteIndented = true }) + "\n";
+ }
+
+ /// The top folders of an addon, the ones gmad's whitelist starts with.
+ private static readonly HashSet AddonFolders = new(
+ WhiteList.Select(rule => rule.TrimStart('!').Split('/')[0]), StringComparer.OrdinalIgnoreCase);
+
+ ///
+ /// The folder whose content goes into an addon: , or the folder inside it that holds
+ /// addon.json or the addon's top folders (lua, materials, models …). An addon zipped as its folder, maybe with a
+ /// readme beside it ("My Addon/lua/…"), is found through folders that hold one subfolder, up to a few levels.
+ ///
+ public static string Root(string folder)
+ {
+ var current = folder;
+ for (var depth = 0; depth < 8; depth++)
+ {
+ if (File.Exists(Path.Combine(current, AddonJson))
+ || Directory.EnumerateDirectories(current).Any(d => AddonFolders.Contains(Path.GetFileName(d))))
+ return current;
+ var folders = Directory.GetDirectories(current);
+ if (folders.Length != 1)
+ break;
+ current = folders[0];
+ }
+ return folder;
+ }
+
+ /// The settings gmad takes from addon.json.
+ internal sealed record AddonSettings(string? Title, string Description, string Type, IReadOnlyList Tags, IReadOnlyList Ignore);
+
+ /// Reads addon.json; a missing file gives the defaults (title from the folder, type "servercontent").
+ internal static AddonSettings ReadSettings(string? jsonPath)
+ {
+ if (jsonPath is null || !File.Exists(jsonPath))
+ return new AddonSettings(null, "Description", "servercontent", [], []);
+ JsonObject json;
+ try
+ {
+ json = JsonNode.Parse(File.ReadAllText(jsonPath), documentOptions: new JsonDocumentOptions
+ {
+ AllowTrailingCommas = true,
+ CommentHandling = JsonCommentHandling.Skip,
+ }) as JsonObject ?? throw new InvalidDataException("addon.json is not a JSON object.");
+ }
+ catch (JsonException ex)
+ {
+ throw new InvalidDataException($"addon.json can't be read: {ex.Message}");
+ }
+
+ static string? Text(JsonNode? node) => node is JsonValue v && v.TryGetValue(out var s) ? s : null;
+ static List Texts(JsonNode? node) => node is JsonArray array ? [.. array.Select(Text).OfType()] : [];
+
+ var title = Text(json["title"]);
+ var type = Text(json["type"]);
+ return new AddonSettings(
+ string.IsNullOrWhiteSpace(title) ? null : title,
+ Text(json["description"]) ?? "Description",
+ string.IsNullOrWhiteSpace(type) ? "servercontent" : type.ToLowerInvariant(),
+ [.. Texts(json["tags"]).Where(t => t.Length > 0).Select(t => t.ToLowerInvariant())],
+ Texts(json["ignore"]));
+ }
+
+ /// True when gmad's whitelist lets (lower case, "/" separated) into an addon.
+ internal static bool IsAllowed(string name)
+ {
+ var valid = false;
+ foreach (var rule in WhiteList)
+ {
+ if (rule.StartsWith('!'))
+ {
+ if (Wildcard(rule[1..], name))
+ valid = false;
+ }
+ else if (!valid)
+ {
+ valid = Wildcard(rule, name);
+ }
+ }
+ return valid;
+ }
+
+ /// True when one of gmad's own rules or an addon.json "ignore" wildcard leaves out.
+ internal static bool IsIgnored(string name, IReadOnlyList ignore)
+ {
+ var lower = name.ToLowerInvariant();
+ if (lower == AddonJson || Wildcard("*thumbs.db", lower) || Wildcard("*desktop.ini", lower) || Wildcard(".git*", lower)
+ || name == ".DS_Store" || Wildcard("*/.DS_Store", name))
+ return true;
+ return ignore.Any(pattern => Wildcard(pattern.Replace('\\', '/'), name));
+ }
+
+ /// Bootil's wildcard test: "*" matches any run of characters (slashes too), everything else itself.
+ internal static bool Wildcard(string pattern, string text)
+ {
+ int p = 0, t = 0, star = -1, mark = 0;
+ while (t < text.Length)
+ {
+ if (p < pattern.Length && pattern[p] == '*')
+ {
+ star = p++;
+ mark = t;
+ }
+ else if (p < pattern.Length && pattern[p] == text[t])
+ {
+ p++;
+ t++;
+ }
+ else if (star >= 0)
+ {
+ p = star + 1;
+ t = ++mark;
+ }
+ else
+ {
+ return false;
+ }
+ }
+ while (p < pattern.Length && pattern[p] == '*')
+ p++;
+ return p == pattern.Length;
+ }
+
+ ///
+ /// Writes an addon of (entry name relative to the addon folder, "/" separated, and the file
+ /// on disk). addon.json among them gives the title, type, tags and ignores; is the
+ /// title without one. Returns the names left out because Garry's Mod would not load them.
+ ///
+ public static IReadOnlyList Write(IReadOnlyList<(string EntryName, string Path)> files, string output, string fallbackTitle,
+ IProgress? progress, CancellationToken cancellationToken)
+ {
+ var json = files.FirstOrDefault(f => string.Equals(f.EntryName, AddonJson, StringComparison.OrdinalIgnoreCase));
+ var settings = ReadSettings(json.Path);
+ var title = settings.Title ?? fallbackTitle;
+
+ var packed = new List<(string Name, string Path, long Size)>();
+ var skipped = new List();
+ var names = new HashSet(StringComparer.Ordinal);
+ foreach (var (entryName, path) in files.OrderBy(f => f.EntryName, StringComparer.Ordinal))
+ {
+ var name = entryName.Replace('\\', '/').TrimStart('/');
+ if (IsIgnored(name, settings.Ignore))
+ continue;
+ var lower = name.ToLowerInvariant();
+ if (!IsAllowed(lower) || !names.Add(lower))
+ {
+ skipped.Add(name);
+ continue;
+ }
+ packed.Add((lower, path, new FileInfo(path).Length));
+ }
+ if (packed.Count == 0)
+ throw new InvalidDataException(
+ "None of these files can go into a Garry's Mod addon. An addon holds lua/, materials/, models/, sound/, maps/ and the like, as Garry's Mod loads them.");
+
+ var description = new JsonObject
+ {
+ ["description"] = settings.Description,
+ ["type"] = settings.Type,
+ ["tags"] = new JsonArray([.. settings.Tags.Select(tag => (JsonNode?)JsonValue.Create(tag))]),
+ }.ToJsonString(new JsonSerializerOptions { WriteIndented = true });
+
+ var total = Math.Max(1, packed.Sum(f => f.Size));
+ long done = 0;
+ var crc = new Crc32();
+ using var file = new FileStream(output, FileMode.CreateNew, FileAccess.Write, FileShare.None, 1 << 16);
+ using var stream = new CrcStream(file, crc);
+ using (var writer = new BinaryWriter(stream, Encoding.UTF8, leaveOpen: true))
+ {
+ writer.Write(Ident);
+ writer.Write(Version);
+ writer.Write(0UL); // SteamID
+ writer.Write((ulong)DateTimeOffset.UtcNow.ToUnixTimeSeconds());
+ writer.Write((byte)0); // no required content
+ WriteString(writer, title);
+ WriteString(writer, description);
+ WriteString(writer, "Author Name"); // as gmad: unused
+ writer.Write(1); // addon version, unused
+
+ var number = 0u;
+ foreach (var (name, path, size) in packed)
+ {
+ writer.Write(++number);
+ WriteString(writer, name);
+ writer.Write(size);
+ writer.Write(FileCrc(path, cancellationToken));
+ }
+ writer.Write(0u);
+ }
+ foreach (var (_, path, size) in packed)
+ {
+ cancellationToken.ThrowIfCancellationRequested();
+ using var source = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite | FileShare.Delete, 1 << 16);
+ var copied = ArchiveFiles.CopyLimited(source, stream, size, cancellationToken);
+ if (copied != size)
+ throw new IOException($"{path} changed while it was packed.");
+ done += size;
+ progress?.Report((double)done / total);
+ }
+ // The CRC of everything written so far, which the game checks after a download.
+ Span tail = stackalloc byte[4];
+ System.Buffers.Binary.BinaryPrimitives.WriteUInt32LittleEndian(tail, crc.Value);
+ file.Write(tail);
+ return skipped;
+ }
+
+ private static void WriteString(BinaryWriter writer, string text)
+ {
+ writer.Write(Encoding.UTF8.GetBytes(text.Replace("\0", "")));
+ writer.Write((byte)0);
+ }
+
+ private static uint FileCrc(string path, CancellationToken cancellationToken)
+ {
+ var crc = new Crc32();
+ var buffer = new byte[1 << 16];
+ using var stream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite | FileShare.Delete, 1 << 16);
+ int read;
+ while ((read = stream.Read(buffer)) > 0)
+ {
+ cancellationToken.ThrowIfCancellationRequested();
+ crc.Append(buffer.AsSpan(0, read));
+ }
+ return crc.Value;
+ }
+}
diff --git a/src/Filee.Engines/Archives/Vpk.cs b/src/Filee.Engines/Archives/Vpk.cs
new file mode 100644
index 000000000..3bfe3fb82
--- /dev/null
+++ b/src/Filee.Engines/Archives/Vpk.cs
@@ -0,0 +1,217 @@
+// Valve pak files (.vpk), the archives the Source engine keeps its games' files in (Half-Life 2, Counter-Strike: Source,
+// Team Fortress 2, the content Garry's Mod mounts). Read by Filee; layout (little endian) as in Valve's vpklib and
+// srctools:
+//
+// signature u32 0x55AA1234, version u32 (1 or 2), tree size u32
+// version 2: file data size, archive MD5 size, other MD5 size, signature size (u32 each; the sections are not read)
+// the tree: extensions, in each its folders, in each its file names (NUL-terminated; an empty string ends each level,
+// " " stands for no extension or no folder); per file a CRC-32 u32, preload size u16, archive index u16,
+// offset u32, length u32 and 0xFFFF, then the preload bytes
+// archive index 0x7FFF: the file's data follows the tree in the directory file itself; any other index n: the data
+// is in "_nnn.vpk" next to "_dir.vpk"
+//
+// A file's content is its preload bytes followed by its data. The PS Vita's .vpk is a ZIP and goes to 7-Zip instead.
+
+using System.Buffers.Binary;
+using System.Text;
+using System.Text.RegularExpressions;
+
+namespace Filee.Engines.Archives;
+
+/// Unpacks Valve pak files.
+internal static partial class Vpk
+{
+ private const uint Signature = 0x55AA1234;
+
+ /// The archive index of data stored in the directory file after the tree.
+ private const ushort InDirectory = 0x7FFF;
+
+ /// The longest name Filee reads from the tree (longer is a damaged file).
+ private const int MaxNameBytes = 4096;
+
+ /// The largest tree Filee reads (Valve's largest, Team Fortress 2's, is about 20 MB).
+ private const uint MaxTreeBytes = 256 * 1024 * 1024;
+
+ /// Data parts are numbered "_000" to "_999".
+ private const int MaxParts = 1000;
+
+ /// One file of a pak.
+ internal sealed record Entry(string Name, uint Crc, byte[] Preload, int Archive, long Offset, long Length)
+ {
+ public long Size => Preload.Length + Length;
+ }
+
+ /// A data part of a split pak ("pak01_003.vpk"), which holds no tree.
+ [GeneratedRegex(@"^(?.+)_(?\d{3})\.vpk$", RegexOptions.IgnoreCase)]
+ private static partial Regex DataPart();
+
+ /// True for a ZIP (a PS Vita package), which 7-Zip reads.
+ public static bool IsZip(string path)
+ {
+ Span start = stackalloc byte[4];
+ using var stream = File.OpenRead(path);
+ return stream.ReadAtLeast(start, 4, throwOnEndOfStream: false) == 4 && start.SequenceEqual("PK\x03\x04"u8);
+ }
+
+ /// Unpacks every file of the pak (its directory file) into .
+ public static void Extract(string archive, string targetDirectory, CancellationToken cancellationToken)
+ {
+ var archiveName = Path.GetFileName(archive);
+ using var directory = new FileStream(archive, FileMode.Open, FileAccess.Read, FileShare.Read, 1 << 16);
+ var (entries, dataStart) = ReadTree(directory, archiveName);
+ // Every name is checked before anything is written: one that leaves the folder stops the whole pak.
+ foreach (var entry in entries)
+ ArchiveSafety.ResolveEntryPath(targetDirectory, entry.Name, archiveName);
+ ArchiveSafety.EnsureRoom(entries.Sum(e => e.Size), targetDirectory, archiveName);
+
+ var parts = new Dictionary();
+ try
+ {
+ foreach (var entry in entries)
+ {
+ cancellationToken.ThrowIfCancellationRequested();
+ var (stream, start) = entry.Archive == InDirectory
+ ? (directory, dataStart + entry.Offset)
+ : (Part(archive, entry.Archive, parts, archiveName), entry.Offset);
+ if (entry.Length > 0 && start + entry.Length > stream.Length)
+ throw ArchiveSafety.Damaged(archiveName, $"{entry.Name} is cut off");
+
+ var path = ArchiveSafety.ResolveEntryPath(targetDirectory, entry.Name, archiveName);
+ Directory.CreateDirectory(Path.GetDirectoryName(path)!);
+ path = ArchiveFiles.UniquePath(path);
+ var crc = new Crc32();
+ using (var output = new CrcStream(new FileStream(path, FileMode.CreateNew, FileAccess.Write, FileShare.None, 1 << 16), crc))
+ {
+ output.Write(entry.Preload);
+ if (entry.Length > 0)
+ {
+ stream.Position = start;
+ ArchiveFiles.CopyLimited(new BoundedStream(stream, entry.Length), output, entry.Length, cancellationToken);
+ }
+ }
+ if (new FileInfo(path).Length != entry.Size)
+ throw ArchiveSafety.Damaged(archiveName, $"{entry.Name} is cut off");
+ if (entry.Crc != 0 && crc.Value != entry.Crc)
+ throw ArchiveSafety.Damaged(archiveName, $"checksum error in {entry.Name}");
+ }
+ }
+ finally
+ {
+ foreach (var part in parts.Values)
+ part.Dispose();
+ }
+ }
+
+ /// The data part of a split pak, opened once.
+ private static FileStream Part(string directoryFile, int index, Dictionary parts, string archiveName)
+ {
+ if (parts.TryGetValue(index, out var open))
+ return open;
+ var folder = Path.GetDirectoryName(Path.GetFullPath(directoryFile))!;
+ var name = Path.GetFileNameWithoutExtension(directoryFile);
+ if (name.EndsWith("_dir", StringComparison.OrdinalIgnoreCase))
+ name = name[..^4];
+ var partName = $"{name}_{index:D3}.vpk";
+ var path = Path.Combine(folder, partName);
+ if (!File.Exists(path))
+ {
+ // Valve's tools write lower case; a copy may have changed it.
+ path = Directory.EnumerateFiles(folder).FirstOrDefault(f => string.Equals(Path.GetFileName(f), partName, StringComparison.OrdinalIgnoreCase))
+ ?? throw new FileNotFoundException($"{archiveName} keeps files in {partName}, which is not next to it.", partName);
+ }
+ var stream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, 1 << 16);
+ parts[index] = stream;
+ return stream;
+ }
+
+ /// Reads the header and the tree; returns the files and where the data in the directory file starts.
+ internal static (List Entries, long DataStart) ReadTree(Stream stream, string archiveName)
+ {
+ Span header = stackalloc byte[12];
+ if (stream.ReadAtLeast(header, 12, throwOnEndOfStream: false) < 12
+ || BinaryPrimitives.ReadUInt32LittleEndian(header) != Signature)
+ {
+ if (DataPart().Match(archiveName) is { Success: true } part)
+ throw new InvalidDataException(
+ $"{archiveName} is part {part.Groups["part"].Value} of a Valve pak; unpack {part.Groups["prefix"].Value}_dir.vpk, which lists its files.");
+ throw new InvalidDataException($"{archiveName} is not a Valve pak (VPK).");
+ }
+ var version = BinaryPrimitives.ReadUInt32LittleEndian(header[4..]);
+ var treeSize = BinaryPrimitives.ReadUInt32LittleEndian(header[8..]);
+ if (version is not (1 or 2))
+ throw new InvalidDataException($"{archiveName} is a VPK of version {version}; Filee reads versions 1 and 2.");
+ var headerSize = version == 2 ? 28 : 12;
+ var dataStart = (long)headerSize + treeSize;
+ if (treeSize > MaxTreeBytes)
+ throw new InvalidDataException($"{archiveName} lists more files than Filee unpacks from one pak.");
+ if (dataStart > stream.Length)
+ throw ArchiveSafety.Damaged(archiveName, "the pak is cut off");
+ stream.Position = headerSize;
+
+ try
+ {
+ var reader = new BinaryReader(stream, Encoding.UTF8, leaveOpen: true);
+ var entries = new List();
+ Span fields = stackalloc byte[18];
+ for (string extension; (extension = ReadString(stream, archiveName)).Length > 0;)
+ {
+ for (string folder; (folder = ReadString(stream, archiveName)).Length > 0;)
+ {
+ for (string file; (file = ReadString(stream, archiveName)).Length > 0;)
+ {
+ if (entries.Count >= ArchiveSafety.MaxEntries)
+ throw new InvalidDataException($"{archiveName} has more than {ArchiveSafety.MaxEntries:N0} files; Filee does not unpack it.");
+ stream.ReadExactly(fields);
+ var crc = BinaryPrimitives.ReadUInt32LittleEndian(fields);
+ var preloadSize = BinaryPrimitives.ReadUInt16LittleEndian(fields[4..]);
+ var archive = BinaryPrimitives.ReadUInt16LittleEndian(fields[6..]);
+ var offset = BinaryPrimitives.ReadUInt32LittleEndian(fields[8..]);
+ var length = BinaryPrimitives.ReadUInt32LittleEndian(fields[12..]);
+ if (BinaryPrimitives.ReadUInt16LittleEndian(fields[16..]) != 0xFFFF
+ || (archive != InDirectory && archive >= MaxParts))
+ throw ArchiveSafety.Damaged(archiveName, "the file list is broken");
+ // The preload bytes are part of the tree: checked against it before they are read.
+ if (stream.Position + preloadSize > dataStart)
+ throw ArchiveSafety.Damaged(archiveName, "the file list is broken");
+ var preload = reader.ReadBytes(preloadSize);
+ if (preload.Length != preloadSize)
+ throw new EndOfStreamException();
+ entries.Add(new Entry(Join(folder, file, extension), crc, preload, archive, offset, length));
+ }
+ }
+ }
+ if (stream.Position > dataStart)
+ throw ArchiveSafety.Damaged(archiveName, "the file list is broken");
+ return (entries, dataStart);
+ }
+ catch (EndOfStreamException)
+ {
+ throw ArchiveSafety.Damaged(archiveName, "the pak is cut off");
+ }
+ }
+
+ /// "folder/name.ext"; " " (Valve's tools) or "" stand for no folder or no extension.
+ private static string Join(string folder, string name, string extension)
+ {
+ folder = folder.Trim() is "" ? "" : folder.Trim().TrimEnd('/') + "/";
+ extension = extension.Trim() is "" ? "" : "." + extension.Trim();
+ return folder + name + extension;
+ }
+
+ /// A NUL-terminated string (Valve writes ASCII; UTF-8 reads it too).
+ private static string ReadString(Stream stream, string archiveName)
+ {
+ var bytes = new List();
+ while (true)
+ {
+ var b = stream.ReadByte();
+ if (b < 0)
+ throw new EndOfStreamException();
+ if (b == 0)
+ return Encoding.UTF8.GetString([.. bytes]);
+ if (bytes.Count >= MaxNameBytes)
+ throw ArchiveSafety.Damaged(archiveName, "a name in the pak is too long");
+ bytes.Add((byte)b);
+ }
+ }
+}
diff --git a/src/Filee.Engines/Models3D/ModelConverter.cs b/src/Filee.Engines/Models3D/ModelConverter.cs
index 03068c326..2afc82ff4 100644
--- a/src/Filee.Engines/Models3D/ModelConverter.cs
+++ b/src/Filee.Engines/Models3D/ModelConverter.cs
@@ -1,10 +1,14 @@
// 3D models in managed code only (SharpGLTF for glTF, Filee's own readers and writers for the rest): GLB, glTF, STL,
-// OBJ, 3MF and PLY into each other through one in-memory model (Model3D), FBX (read only) into any of them, and an
-// isometric preview picture.
+// OBJ, 3MF and PLY into each other through one in-memory model (Model3D), FBX and Source engine models (MDL, with their
+// textures; read only) into any of them, studiomdl data (SMD) to and from them (from an MDL with its bones and weights),
+// an MDL decompiled into the QC and SMDs studiomdl builds it from (SourceDecompiler),
+// and an isometric preview picture.
// See docs/ENGINES.md, "3D models", for units, axes and what each format keeps.
using Filee.Core.Conversion;
+using Filee.Core.Formats;
using Filee.Core.Presets;
+using Filee.Engines.Archives;
using Filee.Engines.Infrastructure;
using Filee.Engines.Magick;
using ImageMagick;
@@ -19,8 +23,8 @@ public sealed class ModelConverter(ILogger? logger = null) : IConverter, IImageT
/// The 3D formats, each readable and writable.
public static readonly string[] Formats = ["glb", "gltf", "stl", "obj", "3mf", "ply"];
- /// Every 3D format that can be read: and FBX.
- public static readonly string[] InputFormats = [.. Formats, "fbx"];
+ /// Every 3D format that can be read: , FBX, Source engine models (MDL) and SMD.
+ public static readonly string[] InputFormats = [.. Formats, "fbx", "mdl", "smd"];
/// Long edge of the preview picture before the preset's resize.
internal const int PreviewSize = 1024;
@@ -39,15 +43,24 @@ public sealed class ModelConverter(ILogger? logger = null) : IConverter, IImageT
public IReadOnlyList Edges { get; } =
[
.. from source in InputFormats from target in Formats where source != target select new ConversionEdge(source, target),
+ // SMD, the source studiomdl compiles: from an MDL with its skeleton, from the others as a static prop.
+ .. from source in InputFormats where source != "smd" select new ConversionEdge(source, "smd"),
+ // A Source model decompiled into the folder studiomdl builds it from: a QC with its SMDs, physics and animations.
+ new ConversionEdge("mdl", FormatRegistry.Folder),
.. from source in InputFormats from target in ImageEncoder.Writable select ImageEncoder.Edge(source, target),
];
- public EngineStatus GetStatus() => EngineStatus.Available("GLB, glTF, STL, OBJ, 3MF, PLY ↔ each other; FBX → each of them",
+ public EngineStatus GetStatus() => EngineStatus.Available("GLB, glTF, STL, OBJ, 3MF, PLY, SMD ↔ each other; FBX, MDL → each of them; MDL → QC and SMDs",
$"{EngineVersions.BuiltIn} · {EngineVersions.Library("SharpGLTF", typeof(SharpGLTF.Schema2.ModelRoot))}");
public Task> ConvertAsync(ConversionStep step, IProgress? progress, CancellationToken cancellationToken) =>
Task.Run>(() =>
{
+ // A Source model keeps its skeleton and weights in an SMD, which Filee's static model has no place for.
+ if (step.From == "mdl" && step.To == "smd")
+ return WriteSkeletalSmd(step, cancellationToken);
+ if (step.From == "mdl" && step.To == FormatRegistry.Folder)
+ return Decompile(step, cancellationToken);
var reading = progress is null ? null : new SubProgress(progress, 0, 0.6);
var model = Read(step.InputPath, step.From, reading, cancellationToken);
_logger.LogDebug("{File}: {Parts} parts, {Vertices} vertices, {Triangles} triangles, {Materials} materials",
@@ -80,6 +93,8 @@ internal static Model3D Read(string path, string format, IProgress? prog
"3mf" => ThreeMfFormat.Read(path, progress, cancellationToken),
"glb" or "gltf" => GltfFormat.Read(path, progress, cancellationToken),
"fbx" => FbxFormat.Read(path, progress, cancellationToken),
+ "mdl" => SourceModel.Load(path, cancellationToken).ToModel3D(path, cancellationToken),
+ "smd" => SmdFormat.Read(path, progress, cancellationToken),
_ => throw new NotSupportedException($"Unknown 3D format '{format}'."),
};
model.Validate();
@@ -97,7 +112,7 @@ internal static void Write(Model3D model, string format, IOutputAllocator output
if (output.Allocate(format) is not { } path)
return;
outputs.Add(path);
- if (!Formats.Contains(format))
+ if (!Formats.Contains(format) && format != "smd")
{
WriteImage(model, format, imageOptions, path, cancellationToken);
return;
@@ -114,6 +129,9 @@ internal static void Write(Model3D model, string format, IOutputAllocator output
case "3mf":
ThreeMfFormat.Write(model, stream, cancellationToken);
break;
+ case "smd":
+ SmdFormat.Write(model, stream, cancellationToken);
+ break;
case "glb":
case "gltf":
GltfFormat.Write(model, stream, binary: format == "glb", cancellationToken);
@@ -121,6 +139,43 @@ internal static void Write(Model3D model, string format, IOutputAllocator output
}
}
+ /// A Source model as a reference SMD with its bones, bind pose and weights.
+ private static IReadOnlyList WriteSkeletalSmd(ConversionStep step, CancellationToken cancellationToken)
+ {
+ var model = SourceModel.Load(step.InputPath, cancellationToken);
+ if (step.Output.Allocate("smd") is not { } path)
+ return [];
+ try
+ {
+ using var stream = new FileStream(path, FileMode.Create, FileAccess.Write, FileShare.None, 1 << 16);
+ SmdFormat.WriteSkeletal(model, stream, cancellationToken);
+ }
+ catch
+ {
+ TryDelete(path);
+ throw;
+ }
+ return [path];
+ }
+
+ /// A Source model decompiled into a new folder (named as the output rule says): its QC and SMDs.
+ private static IReadOnlyList Decompile(ConversionStep step, CancellationToken cancellationToken)
+ {
+ if (step.Output.Allocate("") is not { } folder)
+ return [];
+ Directory.CreateDirectory(folder);
+ try
+ {
+ SourceDecompiler.Decompile(step.InputPath, folder, cancellationToken);
+ }
+ catch
+ {
+ ArchiveFiles.TryDeleteFolder(folder);
+ throw;
+ }
+ return [folder];
+ }
+
/// The .obj, the .mtl next to it (when the model has materials) and the textures next to that.
private static void WriteObj(Model3D model, IOutputAllocator output, List outputs, CancellationToken cancellationToken)
{
diff --git a/src/Filee.Engines/Models3D/SmdFormat.cs b/src/Filee.Engines/Models3D/SmdFormat.cs
new file mode 100644
index 000000000..9d134e4af
--- /dev/null
+++ b/src/Filee.Engines/Models3D/SmdFormat.cs
@@ -0,0 +1,272 @@
+// Studiomdl data (.smd), the text format Valve's studiomdl compiles Source engine models from (Blender Source Tools,
+// Crowbar):
+//
+// version 1
+// nodes one line per bone: index "name" parent (−1 for a root)
+// skeleton / time 0 one line per bone: index, position x y z, rotation x y z (radians), relative to its parent
+// triangles per triangle a material line, then three vertex lines:
+// parent bone, position x y z, normal x y z, u v [, link count, (bone weight) per link]
+// end after each section
+//
+// Positions are in model space, Z up, in inches (Source's unit), triangles counter-clockwise, UVs from the bottom left.
+// Reading takes the triangles of a reference SMD as a static model (an animation SMD has none). Writing from another
+// model gives a static prop: one root bone, every vertex weighted to it. Writing from a Source model (MDL) keeps its
+// bones, bind pose and weights, as a reference mesh Crowbar would decompile.
+
+using System.Globalization;
+using System.Numerics;
+using System.Text;
+
+namespace Filee.Engines.Models3D;
+
+/// Reads and writes studiomdl data files.
+internal static class SmdFormat
+{
+ /// The most triangles Filee reads from one file (about 1 GB of text).
+ private const int MaxTriangles = 10_000_000;
+
+ private static readonly CultureInfo Invariant = CultureInfo.InvariantCulture;
+
+ /// The triangles of a reference SMD, one part per material.
+ public static Model3D Read(string path, IProgress? progress, CancellationToken cancellationToken)
+ {
+ var model = new Model3D();
+ var parts = new Dictionary(StringComparer.Ordinal);
+ var length = Math.Max(1, new FileInfo(path).Length);
+ using var stream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, 1 << 16);
+ using var reader = new StreamReader(stream, Encoding.UTF8, detectEncodingFromByteOrderMarks: true);
+ var lineNumber = 0;
+ string? Next()
+ {
+ while (reader.ReadLine() is { } line)
+ {
+ lineNumber++;
+ var text = line.Trim();
+ if (text.Length > 0 && !text.StartsWith("//", StringComparison.Ordinal))
+ return text;
+ }
+ return null;
+ }
+
+ var triangles = 0;
+ var corners = new int[3];
+ string? section = null;
+ while (Next() is { } line)
+ {
+ if (section is null)
+ {
+ section = line.Split((char[]?)null, 2, StringSplitOptions.RemoveEmptyEntries)[0].ToLowerInvariant();
+ if (section is "version")
+ section = null;
+ continue;
+ }
+ if (line.Equals("end", StringComparison.OrdinalIgnoreCase))
+ {
+ section = null;
+ continue;
+ }
+ if (section != "triangles")
+ continue; // nodes, skeleton, vertex animation: not part of the static mesh
+
+ // A material line, then its triangle's three vertices.
+ cancellationToken.ThrowIfCancellationRequested();
+ if (++triangles > MaxTriangles)
+ throw new InvalidDataException($"The SMD has more than {MaxTriangles:N0} triangles.");
+ var material = line;
+ if (!parts.TryGetValue(material, out var part))
+ {
+ model.Materials.Add(new Material3D { Name = Path.GetFileNameWithoutExtension(material) });
+ part = new MeshPart { Name = Path.GetFileNameWithoutExtension(material), Material = model.Materials.Count - 1, Normals = [], TexCoords = [] };
+ parts[material] = part;
+ model.Parts.Add(part);
+ }
+ for (var k = 0; k < 3; k++)
+ {
+ var vertex = Next() ?? throw new InvalidDataException($"The SMD ends inside a triangle (line {lineNumber}).");
+ var fields = vertex.Split((char[]?)null, StringSplitOptions.RemoveEmptyEntries);
+ if (fields.Length < 9)
+ throw new InvalidDataException($"Line {lineNumber} of the SMD is not a triangle's vertex.");
+ var position = new Vector3(Number(fields[1], lineNumber), Number(fields[2], lineNumber), Number(fields[3], lineNumber));
+ var normal = new Vector3(Number(fields[4], lineNumber), Number(fields[5], lineNumber), Number(fields[6], lineNumber));
+ var uv = new Vector2(Number(fields[7], lineNumber), Number(fields[8], lineNumber));
+ corners[k] = part.AddVertex(MeshMath.ToYUp(position) * SourceModel.MillimetresPerUnit, MeshMath.ToYUp(normal), uv);
+ }
+ part.AddTriangle(corners[0], corners[1], corners[2]);
+ progress?.Report(Math.Min(1, (double)stream.Position / length));
+ }
+ return model;
+ }
+
+ private static float Number(string text, int line) =>
+ float.TryParse(text, NumberStyles.Float, Invariant, out var value) && float.IsFinite(value)
+ ? value
+ : throw new InvalidDataException($"Line {line} of the SMD has \"{text}\" where a number belongs.");
+
+ /// A static prop: one root bone at the origin, every vertex weighted to it.
+ public static void Write(Model3D model, Stream stream, CancellationToken cancellationToken)
+ {
+ using var writer = new StreamWriter(stream, new UTF8Encoding(false), 1 << 16, leaveOpen: true) { NewLine = "\n" };
+ writer.WriteLine("version 1");
+ writer.WriteLine("nodes");
+ writer.WriteLine("0 \"root\" -1");
+ writer.WriteLine("end");
+ writer.WriteLine("skeleton");
+ writer.WriteLine("time 0");
+ writer.WriteLine("0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000");
+ writer.WriteLine("end");
+ writer.WriteLine("triangles");
+ foreach (var part in model.Parts)
+ {
+ if (part.Normals is null)
+ part.ComputeNormals();
+ var material = MaterialName(model.MaterialOf(part), part.Name);
+ var indices = part.Indices;
+ for (var t = 0; t + 2 < indices.Count; t += 3)
+ {
+ if (t % 3000 == 0)
+ cancellationToken.ThrowIfCancellationRequested();
+ writer.WriteLine(material);
+ for (var k = 0; k < 3; k++)
+ {
+ var v = indices[t + k];
+ var position = MeshMath.ToZUp(part.Position(v)) / SourceModel.MillimetresPerUnit;
+ WriteVertex(writer, 0, position, MeshMath.ToZUp(part.Normal(v)), part.TexCoord(v));
+ writer.WriteLine(" 1 0 1.000000");
+ }
+ }
+ }
+ writer.WriteLine("end");
+ }
+
+ /// A Source model as a reference SMD: its bones in their bind pose, its triangles with their weights.
+ public static void WriteSkeletal(SourceModel model, Stream stream, CancellationToken cancellationToken) =>
+ WriteSkeletal(model, model.Meshes, stream, cancellationToken);
+
+ /// A reference SMD of (some of 's) with its bones and weights.
+ public static void WriteSkeletal(SourceModel model, IReadOnlyList meshes, Stream stream,
+ CancellationToken cancellationToken)
+ {
+ using var writer = new StreamWriter(stream, new UTF8Encoding(false), 1 << 16, leaveOpen: true) { NewLine = "\n" };
+ var bones = WriteBindPose(writer, model.Bones);
+ writer.WriteLine("triangles");
+ foreach (var mesh in meshes)
+ {
+ cancellationToken.ThrowIfCancellationRequested();
+ var material = Clean(mesh.Material);
+ // Source's triangles are clockwise; an SMD's counter-clockwise, as Crowbar writes them.
+ for (var t = 0; t + 2 < mesh.Triangles.Count; t += 3)
+ {
+ writer.WriteLine(material);
+ foreach (var index in new[] { mesh.Triangles[t + 2], mesh.Triangles[t + 1], mesh.Triangles[t] })
+ {
+ var v = model.Vertices[index];
+ var bone = Math.Clamp(v.Bone0, 0, bones.Count - 1);
+ WriteVertex(writer, bone, v.Position, v.Normal, new Vector2(v.Uv.X, 1 - v.Uv.Y));
+ var links = new StringBuilder();
+ var count = 0;
+ foreach (var (b, w) in new[] { (v.Bone0, v.Weight0), (v.Bone1, v.Weight1), (v.Bone2, v.Weight2) }.Take(v.BoneCount))
+ {
+ if (b < 0 || b >= bones.Count)
+ continue;
+ links.Append(Invariant, $" {b} {w:F6}");
+ count++;
+ }
+ writer.WriteLine(count > 0 ? $" {count}{links}" : $" 1 {bone} 1.000000");
+ }
+ }
+ }
+ writer.WriteLine("end");
+ }
+
+ /// An animation SMD: the bones, then every frame's pose of every bone.
+ public static void WriteAnimation(IReadOnlyList bones, IReadOnlyList frames, Stream stream,
+ CancellationToken cancellationToken)
+ {
+ using var writer = new StreamWriter(stream, new UTF8Encoding(false), 1 << 16, leaveOpen: true) { NewLine = "\n" };
+ WriteNodes(writer, bones);
+ writer.WriteLine("skeleton");
+ for (var f = 0; f < frames.Count; f++)
+ {
+ cancellationToken.ThrowIfCancellationRequested();
+ writer.WriteLine(string.Create(Invariant, $"time {f}"));
+ for (var i = 0; i < bones.Count; i++)
+ WriteBone(writer, i, frames[f][i].Position, frames[f][i].Rotation);
+ }
+ writer.WriteLine("end");
+ }
+
+ ///
+ /// A physics SMD: the bones in their bind pose, then each convex piece's triangles (material "phy"), every vertex
+ /// on its piece's bone, with the face's normal.
+ ///
+ public static void WritePhysics(IReadOnlyList bones,
+ IEnumerable<(int Bone, IReadOnlyList Points, IReadOnlyList Triangles)> pieces, Stream stream,
+ CancellationToken cancellationToken)
+ {
+ using var writer = new StreamWriter(stream, new UTF8Encoding(false), 1 << 16, leaveOpen: true) { NewLine = "\n" };
+ var skeleton = WriteBindPose(writer, bones);
+ writer.WriteLine("triangles");
+ foreach (var (pieceBone, points, triangles) in pieces)
+ {
+ cancellationToken.ThrowIfCancellationRequested();
+ var bone = Math.Clamp(pieceBone, 0, skeleton.Count - 1);
+ for (var t = 0; t + 2 < triangles.Count; t += 3)
+ {
+ var (a, b, c) = (points[triangles[t]], points[triangles[t + 1]], points[triangles[t + 2]]);
+ var normal = Vector3.Cross(b - a, c - a);
+ normal = normal.LengthSquared() > 0 ? Vector3.Normalize(normal) : Vector3.UnitZ;
+ writer.WriteLine("phy");
+ foreach (var p in new[] { a, b, c })
+ {
+ WriteVertex(writer, bone, p, normal, Vector2.Zero);
+ writer.WriteLine(string.Create(Invariant, $" 1 {bone} 1.000000"));
+ }
+ }
+ }
+ writer.WriteLine("end");
+ }
+
+ /// The nodes and the bind pose ("time 0"); a model without bones gets one root bone. Returns the bones written.
+ private static IReadOnlyList WriteBindPose(StreamWriter writer, IReadOnlyList modelBones)
+ {
+ IReadOnlyList bones = modelBones.Count > 0 ? modelBones : [new SourceModel.Bone("root", -1, Vector3.Zero, Vector3.Zero)];
+ WriteNodes(writer, bones);
+ writer.WriteLine("skeleton");
+ writer.WriteLine("time 0");
+ for (var i = 0; i < bones.Count; i++)
+ WriteBone(writer, i, bones[i].Position, bones[i].Rotation);
+ writer.WriteLine("end");
+ return bones;
+ }
+
+ private static void WriteNodes(StreamWriter writer, IReadOnlyList bones)
+ {
+ writer.WriteLine("version 1");
+ writer.WriteLine("nodes");
+ for (var i = 0; i < bones.Count; i++)
+ writer.WriteLine($"{i} \"{bones[i].Name.Replace('"', '\'')}\" {bones[i].Parent}");
+ writer.WriteLine("end");
+ }
+
+ private static void WriteBone(StreamWriter writer, int index, Vector3 p, Vector3 r) =>
+ writer.WriteLine(string.Create(Invariant, $"{index} {p.X:F6} {p.Y:F6} {p.Z:F6} {r.X:F6} {r.Y:F6} {r.Z:F6}"));
+
+ private static void WriteVertex(StreamWriter writer, int bone, Vector3 p, Vector3 n, Vector2 uv) =>
+ writer.Write(string.Create(Invariant, $"{bone} {p.X:F6} {p.Y:F6} {p.Z:F6} {n.X:F6} {n.Y:F6} {n.Z:F6} {uv.X:F6} {uv.Y:F6}"));
+
+ /// The material line: studiomdl looks the name up as a .vmt, so it is the material's or texture's name.
+ private static string MaterialName(Material3D? material, string partName)
+ {
+ var name = material?.Name is { Length: > 0 } named ? named
+ : material?.BaseColorTexture?.Name is { Length: > 0 } texture ? texture
+ : partName;
+ return Clean(name);
+ }
+
+ /// One word without spaces or quotes; "default" when nothing is left.
+ private static string Clean(string name)
+ {
+ var clean = new string([.. name.Trim().Select(c => char.IsWhiteSpace(c) || c == '"' ? '_' : c)]);
+ return clean.Length == 0 ? "default" : clean;
+ }
+}
diff --git a/src/Filee.Engines/Models3D/SourceAnimation.cs b/src/Filee.Engines/Models3D/SourceAnimation.cs
new file mode 100644
index 000000000..13ee0d759
--- /dev/null
+++ b/src/Filee.Engines/Models3D/SourceAnimation.cs
@@ -0,0 +1,293 @@
+// The frames of a Source engine animation (#1526), decoded from the .mdl (or the .ani file its animation blocks are in)
+// into each bone's position and rotation relative to its parent, as an animation SMD stores them. Layout (studio.h):
+//
+// mstudioanim_t per animated bone: bone (byte), flags (byte), offset to the next bone's entry (short, 0 for the
+// last), then its data: a rotation (Quaternion64 with flag 0x20, Quaternion48 with 0x02, or three
+// run-length encoded tracks with 0x08), then a position (Vector48 with 0x01, or three tracks with 0x04)
+// tracks three offsets (shorts, relative to the first; 0 for a track that stays 0) to runs of
+// (valid, total) byte pairs, each followed by its valid values (shorts); frames after the valid ones
+// repeat the last. A value times the bone's scale is added to its bind pose (not for delta animations)
+// Quaternion48 x and y in 16 bits, z in 15, the sign of w in 1; Quaternion64 x, y and z in 21 bits each
+// Vector48 three half-precision floats
+// sections long animations are cut into sections of a number of frames, each with its own block and offset
+// movements 44 bytes each: end frame, motion flags, start and end speed, angle, direction, position reached;
+// studiomdl took this motion out of the root bones, so it is added back
+//
+// Bones an animation does not touch keep their bind pose (zero for a delta animation).
+
+using System.Numerics;
+
+namespace Filee.Engines.Models3D;
+
+/// Decodes the frames of a Source model's animations.
+internal static class SourceAnimation
+{
+ /// The most frames Filee decodes from one animation.
+ public const int MaxFrames = 100_000;
+
+ /// The most bone entries one frame may chain (each bone at most once, in practice).
+ private const int MaxEntries = SourceModel.MaxBones * 4;
+
+ private const int MaxMovements = 4096;
+
+ /// A bone's pose: position and rotation (radians, Source's Euler order) relative to its parent.
+ internal readonly record struct Pose(Vector3 Position, Vector3 Rotation);
+
+ // studio.h's mstudioanim_t flags and STUDIO_DELTA.
+ private const int RawPosition = 0x01, RawRotation = 0x02, AnimatedPosition = 0x04, AnimatedRotation = 0x08, RawRotation64 = 0x20;
+ public const int DeltaFlag = 0x04;
+
+ /// The motion flags of a movement, in the order the QC writes them.
+ public static readonly (int Flag, string Name)[] MotionFlags =
+ [
+ (0x0001, "X"), (0x0002, "Y"), (0x0004, "Z"), (0x0008, "XR"), (0x0010, "YR"), (0x0020, "ZR"),
+ (0x0040, "LX"), (0x0080, "LY"), (0x0100, "LZ"), (0x0200, "LXR"), (0x0400, "LYR"), (0x0800, "LZR"),
+ ];
+
+ ///
+ /// The poses of every bone in every frame of . is the model's .ani
+ /// file, needed when the animation lies in one of its blocks.
+ ///
+ public static List Decode(SourceModelDetails details, SourceModelDetails.Animation animation, SourceBytes? ani,
+ CancellationToken cancellationToken)
+ {
+ var mdl = details.Model.Mdl;
+ var bones = details.Model.Bones;
+ var delta = (animation.Flags & DeltaFlag) != 0;
+ var at = animation.Offset;
+ var sectionFrames = mdl.I32(at + 84);
+ if (sectionFrames < 0)
+ throw new InvalidDataException($"{mdl.File} is damaged (an animation claims {sectionFrames} frames a section).");
+ var movements = ReadMovements(mdl, at);
+ var frames = new List(animation.Frames);
+ for (var frame = 0; frame < animation.Frames; frame++)
+ {
+ cancellationToken.ThrowIfCancellationRequested();
+ int block, index, local;
+ if (sectionFrames > 0)
+ {
+ var section = frame / sectionFrames;
+ local = frame - section * sectionFrames;
+ var entry = at + mdl.I32(at + 80) + section * 8;
+ (block, index) = (mdl.I32(entry), mdl.I32(entry + 4));
+ }
+ else
+ {
+ (block, index, local) = (mdl.I32(at + 52), mdl.I32(at + 56), frame);
+ }
+
+ var pose = new Pose[bones.Count];
+ for (var b = 0; b < bones.Count; b++)
+ pose[b] = delta ? default : new Pose(bones[b].Position, bones[b].Rotation);
+ var (data, start) = Locate(details, ani, block, at, index);
+ if (start >= 0)
+ DecodeFrame(details, data, start, local, delta, pose);
+ AddMovement(movements, frame, bones, pose);
+ frames.Add(pose);
+ }
+ return frames;
+ }
+
+ ///
+ /// The file and offset an animation's data starts at: the .mdl (block 0, relative to the description) or a block
+ /// of the .ani; −1 when the animation has no data (every bone at its bind pose).
+ ///
+ private static (SourceBytes Data, int Start) Locate(SourceModelDetails details, SourceBytes? ani, int block, int description, int index)
+ {
+ var mdl = details.Model.Mdl;
+ if (block == 0)
+ return index == 0 ? (mdl, -1) : (mdl, description + index);
+ if (block < 0 || block >= details.AnimationBlocks.Count)
+ throw new InvalidDataException($"{mdl.File} is damaged (an animation is in block {block}).");
+ if (ani is null)
+ throw new FileNotFoundException($"{mdl.File} keeps animations in {Path.GetFileNameWithoutExtension(mdl.File)}.ani, which is not next to it.");
+ var (blockStart, blockEnd) = details.AnimationBlocks[block];
+ if (blockStart < 0 || blockEnd > ani.Data.Length || index < 0 || (long)blockStart + index >= blockEnd)
+ throw ani.Damaged("an animation block");
+ return (ani, blockStart + index);
+ }
+
+ /// Applies the animated bones' entries at for frame .
+ private static void DecodeFrame(SourceModelDetails details, SourceBytes data, int start, int frame, bool delta, Pose[] pose)
+ {
+ var bones = details.Model.Bones;
+ var entry = start;
+ for (var n = 0; n < MaxEntries; n++)
+ {
+ var bone = data.Byte(entry);
+ var flags = data.Byte(entry + 1);
+ var next = data.I16(entry + 2);
+ // Bone 255 marks an animation with no animated bones (studiomdl writes it for a single still frame).
+ if (bone == 255)
+ return;
+ if (bone >= bones.Count)
+ throw data.Damaged("an animated bone");
+ var (positionScale, rotationScale) = details.BoneScales[bone];
+ var bind = bones[bone];
+ var (position, rotation) = (pose[bone].Position, pose[bone].Rotation);
+ var at = entry + 4;
+ if ((flags & RawRotation64) != 0)
+ {
+ rotation = ToEuler(Quaternion64(data, at));
+ at += 8;
+ }
+ if ((flags & RawRotation) != 0)
+ {
+ rotation = ToEuler(Quaternion48(data, at));
+ at += 6;
+ }
+ if ((flags & AnimatedRotation) != 0)
+ {
+ rotation = (delta ? Vector3.Zero : bind.Rotation) + Tracks(data, at, frame) * rotationScale;
+ at += 6;
+ }
+ if ((flags & RawPosition) != 0)
+ {
+ position = new Vector3(data.F16(at), data.F16(at + 2), data.F16(at + 4));
+ at += 6;
+ }
+ if ((flags & AnimatedPosition) != 0)
+ position = (delta ? Vector3.Zero : bind.Position) + Tracks(data, at, frame) * positionScale;
+ pose[bone] = new Pose(position, rotation);
+ if (next == 0)
+ return;
+ if (next < 0)
+ throw data.Damaged("an animated bone");
+ entry += next;
+ }
+ throw new InvalidDataException($"{data.File} is damaged (an animation frame never ends).");
+ }
+
+ /// The three run-length encoded tracks whose offsets are at , at .
+ private static Vector3 Tracks(SourceBytes data, int at, int frame)
+ {
+ float Track(int component)
+ {
+ var offset = data.I16(at + component * 2);
+ return offset == 0 ? 0 : Value(data, at + offset, frame);
+ }
+ return new Vector3(Track(0), Track(1), Track(2));
+ }
+
+ /// One track's value at : runs of (valid, total) and valid values, the last repeated.
+ private static float Value(SourceBytes data, int at, int frame)
+ {
+ var k = frame;
+ for (var runs = 0; runs <= MaxFrames; runs++)
+ {
+ var valid = data.Byte(at);
+ var total = data.Byte(at + 1);
+ if (total == 0)
+ return 0;
+ if (total > k)
+ return data.I16(at + 2 * (valid > k ? k + 1 : valid));
+ k -= total;
+ at += (valid + 1) * 2;
+ }
+ throw new InvalidDataException($"{data.File} is damaged (an animation track never ends).");
+ }
+
+ private static Quaternion Quaternion48(SourceBytes data, int at)
+ {
+ var x = (data.U16(at) - 32768) / 32768f;
+ var y = (data.U16(at + 2) - 32768) / 32768f;
+ var zw = data.U16(at + 4);
+ var z = ((zw & 0x7FFF) - 16384) / 16384f;
+ return WithW(x, y, z, (zw & 0x8000) != 0);
+ }
+
+ private static Quaternion Quaternion64(SourceBytes data, int at)
+ {
+ var bits = data.U64(at);
+ float Part(int shift) => ((long)((bits >> shift) & 0x1FFFFF) - 1048576) / 1048576.5f;
+ return WithW(Part(0), Part(21), Part(42), (bits >> 63) != 0);
+ }
+
+ private static Quaternion WithW(float x, float y, float z, bool negative)
+ {
+ var w = MathF.Sqrt(MathF.Max(0, 1 - x * x - y * y - z * z));
+ return new Quaternion(x, y, z, negative ? -w : w);
+ }
+
+ ///
+ /// A rotation as Source's Euler angles (radians): the rotation about X, then Y, then Z (z yaw, y pitch, x roll), as
+ /// bones and SMDs store them.
+ ///
+ public static Vector3 ToEuler(Quaternion q)
+ {
+ var (x, y, z, w) = (q.X, q.Y, q.Z, q.W);
+ var m00 = 1 - 2 * (y * y + z * z);
+ var m10 = 2 * (x * y + w * z);
+ var m20 = 2 * (x * z - w * y);
+ var m21 = 2 * (y * z + w * x);
+ var m22 = 1 - 2 * (x * x + y * y);
+ var cosPitch = MathF.Sqrt(m00 * m00 + m10 * m10);
+ if (cosPitch > 1e-6f)
+ return new Vector3(MathF.Atan2(m21, m22), MathF.Atan2(-m20, cosPitch), MathF.Atan2(m10, m00));
+ // Looking straight up or down: the roll takes all of the rotation about the vertical.
+ var m11 = 1 - 2 * (x * x + z * z);
+ var m12 = 2 * (y * z - w * x);
+ return new Vector3(MathF.Atan2(-m12, m11), MathF.Atan2(-m20, cosPitch), 0);
+ }
+
+ /// Source's Euler angles (radians) as a quaternion, the inverse of .
+ public static Quaternion FromEuler(Vector3 angles)
+ {
+ var (sr, cr) = MathF.SinCos(angles.X / 2);
+ var (sp, cp) = MathF.SinCos(angles.Y / 2);
+ var (sy, cy) = MathF.SinCos(angles.Z / 2);
+ return new Quaternion(sr * cp * cy - cr * sp * sy, cr * sp * cy + sr * cp * sy, cr * cp * sy - sr * sp * cy, cr * cp * cy + sr * sp * sy);
+ }
+
+ internal readonly record struct Movement(int EndFrame, int Flags, float StartSpeed, float EndSpeed, Vector3 Position);
+
+ /// The movements (linear motion studiomdl took out of the root bones) of the animation at .
+ public static List ReadMovements(SourceBytes mdl, int at)
+ {
+ var count = mdl.Count(at + 20, MaxMovements, "movements");
+ var start = at + mdl.I32(at + 24);
+ var movements = new List(count);
+ for (var i = 0; i < count; i++)
+ {
+ var m = start + i * 44;
+ movements.Add(new Movement(mdl.I32(m), mdl.I32(m + 4), mdl.F32(m + 8), mdl.F32(m + 12), mdl.Vector(m + 32)));
+ }
+ return movements;
+ }
+
+ ///
+ /// Adds the motion reached by to the root bones: the position the previous movements
+ /// reached, plus the part of this movement's way covered so far (its speed changing evenly from start to end).
+ ///
+ private static void AddMovement(List movements, int frame, IReadOnlyList bones, Pose[] pose)
+ {
+ if (movements.Count == 0)
+ return;
+ var offset = Vector3.Zero;
+ var previous = 0;
+ foreach (var movement in movements)
+ {
+ if (movement.EndFrame >= frame)
+ {
+ var span = movement.EndFrame - previous;
+ var t = span > 0 ? (frame - previous) / (float)span : 1;
+ // Distance at time t with the speed going from start to end, as a share of the whole movement's.
+ var whole = movement.StartSpeed + 0.5f * (movement.EndSpeed - movement.StartSpeed);
+ var share = MathF.Abs(whole) > 1e-6f
+ ? (movement.StartSpeed * t + 0.5f * (movement.EndSpeed - movement.StartSpeed) * t * t) / whole
+ : t;
+ offset += (movement.Position - offset) * share;
+ break;
+ }
+ previous = movement.EndFrame;
+ offset = movement.Position;
+ }
+ if (offset == Vector3.Zero)
+ return;
+ for (var b = 0; b < bones.Count; b++)
+ {
+ if (bones[b].Parent < 0)
+ pose[b] = pose[b] with { Position = pose[b].Position + offset };
+ }
+ }
+}
diff --git a/src/Filee.Engines/Models3D/SourceBytes.cs b/src/Filee.Engines/Models3D/SourceBytes.cs
new file mode 100644
index 000000000..7e23b28e2
--- /dev/null
+++ b/src/Filee.Engines/Models3D/SourceBytes.cs
@@ -0,0 +1,71 @@
+// The bytes of a Source engine model file (.mdl, .vvd, .vtx, .phy, .ani) with bounds-checked little-endian reads: an
+// offset or count that points outside the file is a damaged file, never a crash or a read of other memory.
+
+using System.Buffers.Binary;
+using System.Numerics;
+using System.Text;
+
+namespace Filee.Engines.Models3D;
+
+/// A Source engine file's bytes with bounds-checked little-endian reads.
+internal sealed class SourceBytes(byte[] data, string file)
+{
+ /// The largest Source engine file Filee reads (a damaged or made-up file otherwise).
+ public const long MaxFileSize = 256L * 1024 * 1024;
+
+ public byte[] Data { get; } = data;
+ public string File { get; } = file;
+
+ /// Reads , refusing files over .
+ public static SourceBytes Read(string path)
+ {
+ var length = new FileInfo(path).Length;
+ if (length > MaxFileSize)
+ throw new InvalidDataException($"{Path.GetFileName(path)} is too large to be a Source engine model file.");
+ return new SourceBytes(System.IO.File.ReadAllBytes(path), Path.GetFileName(path));
+ }
+
+ public InvalidDataException Damaged(string what) => new($"{File} is damaged ({what} points outside the file).");
+
+ private ReadOnlySpan At(int offset, int length) =>
+ offset >= 0 && (long)offset + length <= Data.Length ? Data.AsSpan(offset, length) : throw Damaged("an offset");
+
+ /// Whether items of bytes from lie in the file.
+ public bool Fits(long offset, long count, int size) => offset >= 0 && count >= 0 && offset + count * size <= Data.Length;
+
+ public int I32(int offset) => BinaryPrimitives.ReadInt32LittleEndian(At(offset, 4));
+ public uint U32(int offset) => BinaryPrimitives.ReadUInt32LittleEndian(At(offset, 4));
+ public ulong U64(int offset) => BinaryPrimitives.ReadUInt64LittleEndian(At(offset, 8));
+ public int I16(int offset) => BinaryPrimitives.ReadInt16LittleEndian(At(offset, 2));
+ public int U16(int offset) => BinaryPrimitives.ReadUInt16LittleEndian(At(offset, 2));
+ public byte Byte(int offset) => At(offset, 1)[0];
+ public float F32(int offset) => BinaryPrimitives.ReadSingleLittleEndian(At(offset, 4));
+ public float F16(int offset) => (float)BinaryPrimitives.ReadHalfLittleEndian(At(offset, 2));
+ public Vector3 Vector(int offset) => new(F32(offset), F32(offset + 4), F32(offset + 8));
+
+ /// A count at , at most .
+ public int Count(int offset, int max, string what)
+ {
+ var count = I32(offset);
+ if (count < 0 || count > max)
+ throw new InvalidDataException($"{File} is damaged (it claims {count} {what}).");
+ return count;
+ }
+
+ /// A NUL-terminated name at , at most bytes.
+ public string String(int offset, int max = 1024)
+ {
+ if (offset < 0 || offset >= Data.Length)
+ throw Damaged("a name");
+ var span = Data.AsSpan(offset, Math.Min(max, Data.Length - offset));
+ var end = span.IndexOf((byte)0);
+ return Encoding.UTF8.GetString(end < 0 ? span : span[..end]);
+ }
+
+ /// A name at + the relative offset stored at ; "" for offset 0.
+ public string RelativeString(int baseOffset, int field, int max = 1024)
+ {
+ var relative = I32(field);
+ return relative == 0 ? "" : String(baseOffset + relative, max);
+ }
+}
diff --git a/src/Filee.Engines/Models3D/SourceDecompiler.cs b/src/Filee.Engines/Models3D/SourceDecompiler.cs
new file mode 100644
index 000000000..6360bc8cf
--- /dev/null
+++ b/src/Filee.Engines/Models3D/SourceDecompiler.cs
@@ -0,0 +1,494 @@
+// Decompiling a Source engine model (#1526): the .mdl with its .vvd, .vtx, .phy and .ani becomes the folder studiomdl
+// could build it again from, as Crowbar decompiles it:
+//
+// .qc $modelname, body groups, LODs, materials and skins, attachments, hitboxes, pose
+// parameters, sequences with their events, collision model and settings, key values
+// .smd one reference SMD per body group model (its bones, bind pose and weights)
+// _lod.smd a model's LOD, when it differs from the one before
+// _physics.smd the convex pieces of the .phy
+// _anims/.smd one animation SMD per animation, every frame of every bone
+//
+// Not written: flexes (facial animation, .vta), eyeballs, IK rules and chains, bone controllers, and the procedural
+// bones of a .vrd; a model that uses them still compiles from the QC, without them.
+
+using System.Globalization;
+using System.Numerics;
+using System.Text;
+
+namespace Filee.Engines.Models3D;
+
+/// Decompiles a Source engine model into a QC and its SMDs.
+internal static class SourceDecompiler
+{
+ private static readonly CultureInfo Invariant = CultureInfo.InvariantCulture;
+
+ // studiohdr_t flags (studio.h) and the QC commands that set them.
+ private static readonly (int Flag, string Command)[] ModelFlags =
+ [
+ (0x00010, "$staticprop"), (0x00004, "$opaque"), (0x00008, "$mostlyopaque"), (0x10000, "$ambientboost"),
+ (0x20000, "$donotcastshadows"), (0x40000, "$casttextureshadows"),
+ ];
+
+ private const int StaticProp = 0x10;
+ private const int AutogeneratedHitboxes = 0x1;
+ private const int ConstantDirectionalLight = 0x2000;
+
+ // mstudioseqdesc_t flags (studio.h) and the sequence options that set them.
+ private static readonly (int Flag, string Option)[] SequenceFlags =
+ [
+ (0x0001, "loop"), (0x0002, "snap"), (0x0004, "delta"), (0x0008, "autoplay"), (0x0010, "post"), (0x0100, "realtime"),
+ (0x0400, "hidden"), (0x4000, "worldspace"),
+ ];
+
+ /// An event whose type has this bit is named, not numbered (NEW_EVENT_STYLE).
+ private const int NamedEvent = 0x400;
+
+ /// Writes the decompiled into , which exists and is empty.
+ public static void Decompile(string mdlPath, string folder, CancellationToken cancellationToken)
+ {
+ var model = SourceModel.Load(mdlPath, cancellationToken);
+ var details = SourceModelDetails.Read(model, cancellationToken);
+ var name = SafeName(Path.GetFileNameWithoutExtension(mdlPath));
+ var basePath = Path.Combine(Path.GetDirectoryName(Path.GetFullPath(mdlPath))!, Path.GetFileNameWithoutExtension(mdlPath));
+ var files = new HashSet(StringComparer.OrdinalIgnoreCase) { name + ".qc" };
+ string Unique(string stem, string extension = ".smd")
+ {
+ var file = stem + extension;
+ for (var n = 2; !files.Add(file); n++)
+ file = $"{stem}_{n}{extension}";
+ return file;
+ }
+
+ // The commands in Crowbar's order and layout, so a QC from Filee reads like one from Crowbar.
+ var qc = new StringBuilder();
+ qc.Append(Invariant, $"// Decompiled by Filee from {Path.GetFileName(mdlPath)} (version {details.Version}).\n\n");
+ qc.Append(Invariant, $"$modelname {Quote(model.Name)}\n\n");
+ if ((details.Flags & StaticProp) != 0)
+ qc.Append("$staticprop\n\n");
+
+ var references = WriteBodyGroups(qc, details, folder, Unique, cancellationToken);
+ WriteLods(qc, details, references, folder, Unique, cancellationToken);
+ WriteModelSettings(qc, details);
+ WriteMaterials(qc, details);
+ WriteAttachments(qc, details);
+ qc.Append(Invariant, $"$cbox {Vector(details.ViewMin)} {Vector(details.ViewMax)}\n\n");
+ qc.Append(Invariant, $"$bbox {Vector(details.HullMin)} {Vector(details.HullMax)}\n\n");
+ WriteDefineBones(qc, details);
+ WriteHitboxes(qc, details);
+ foreach (var pose in details.PoseParameters)
+ {
+ qc.Append(Invariant, $"$poseparameter {Quote(pose.Name)} {Number(pose.Start)} {Number(pose.End)}");
+ qc.Append((pose.Flags & 1) != 0 ? $" loop {Number(pose.Loop)}\n" : "\n");
+ }
+ if (details.PoseParameters.Count > 0)
+ qc.Append('\n');
+
+ var aniPath = SourceModel.Companion(basePath, [".ani"]);
+ var ani = aniPath is null ? null : SourceBytes.Read(aniPath);
+ WriteAnimations(qc, details, ani, name, folder, cancellationToken);
+
+ // studiomdl puts "models/" in front of an included model itself.
+ foreach (var include in details.IncludeModels)
+ qc.Append(Invariant, $"$includemodel {Quote(WithoutModelsFolder(include))}\n\n");
+
+ var phyPath = SourceModel.Companion(basePath, [".phy"]);
+ if (phyPath is not null)
+ WritePhysics(qc, details, SourcePhysics.Read(phyPath), Unique(name + "_physics"), folder, cancellationToken);
+
+ if (details.KeyValues.Length > 0)
+ qc.Append("$keyvalues\n{\n").Append(Indent(details.KeyValues)).Append("\n}\n");
+
+ File.WriteAllText(Path.Combine(folder, name + ".qc"), qc.ToString(), new UTF8Encoding(false));
+ }
+
+ ///
+ /// $surfaceprop, $contents, $illumposition, $eyeposition and the model's flags. studiomdl reads the illumination and
+ /// eye positions turned a quarter about Z (it stores QC (x, y) as (−y, x)), so they are turned back.
+ ///
+ private static void WriteModelSettings(StringBuilder qc, SourceModelDetails details)
+ {
+ if (details.SurfaceProperty.Length > 0)
+ qc.Append(Invariant, $"$surfaceprop {Quote(details.SurfaceProperty)}\n\n");
+ qc.Append("$contents ").AppendJoin(' ', ContentsNames(details.Contents).Select(Quote)).Append("\n\n");
+ qc.Append(Invariant, $"$illumposition {Vector(AsWritten(details.IlluminationPosition))}\n\n");
+ if (details.EyePosition != Vector3.Zero)
+ qc.Append(Invariant, $"$eyeposition {Vector(AsWritten(details.EyePosition))}\n\n");
+ var flags = ModelFlags.Where(f => f.Flag != StaticProp && (details.Flags & f.Flag) != 0).ToList();
+ foreach (var (_, command) in flags)
+ qc.Append(command).Append('\n');
+ if ((details.Flags & ConstantDirectionalLight) != 0)
+ qc.Append(Invariant, $"$constantdirectionallight {Number(details.DirectionalLightDot / 255f)}\n");
+ if (flags.Count > 0 || (details.Flags & ConstantDirectionalLight) != 0)
+ qc.Append('\n');
+ }
+
+ /// A position as the QC wrote it, before studiomdl turned it: (x, y, z) stored is (y, −x, z) in the QC.
+ private static Vector3 AsWritten(Vector3 stored) => new(stored.Y, -stored.X, stored.Z);
+
+ // bspflags.h's CONTENTS_ values that $contents names.
+ private static readonly (int Flag, string Name)[] Contents =
+ [
+ (0x1, "solid"), (0x8, "grate"), (0x2000000, "monster"), (0x20000000, "ladder"),
+ ];
+
+ private static IEnumerable ContentsNames(int contents) =>
+ contents == 0 ? ["notsolid"] : Contents.Where(c => (contents & c.Flag) != 0).Select(c => c.Name).DefaultIfEmpty("solid");
+
+ private static string WithoutModelsFolder(string path)
+ {
+ var normal = path.Replace('\\', '/');
+ return normal.StartsWith("models/", StringComparison.OrdinalIgnoreCase) ? normal["models/".Length..] : normal;
+ }
+
+ ///
+ /// A $definebone line for every bone, as Crowbar writes them: its parent, its position and its rotation as pitch, yaw
+ /// and roll in degrees (the rotations about Y, Z and X), and no fix-up.
+ ///
+ private static void WriteDefineBones(StringBuilder qc, SourceModelDetails details)
+ {
+ var bones = details.Model.Bones;
+ foreach (var bone in bones)
+ {
+ var parent = bone.Parent >= 0 && bone.Parent < bones.Count ? bones[bone.Parent].Name : "";
+ var r = bone.Rotation * (180 / MathF.PI);
+ qc.Append(Invariant, $"$definebone {Quote(bone.Name)} {Quote(parent)} {Vector(bone.Position)} {Number(r.Y)} {Number(r.Z)} {Number(r.X)} 0 0 0 0 0 0\n");
+ }
+ if (bones.Count > 0)
+ qc.Append("\n\n");
+ }
+
+ ///
+ /// The $bodygroup blocks and a reference SMD for every body group model with triangles. Returns the file name
+ /// written for each body model (null for a blank one), by body part and model.
+ ///
+ private static List> WriteBodyGroups(StringBuilder qc, SourceModelDetails details, string folder,
+ Func unique, CancellationToken cancellationToken)
+ {
+ var references = new List>();
+ foreach (var part in details.BodyParts)
+ {
+ var files = new List();
+ qc.Append(Invariant, $"$bodygroup {Quote(part.Name)}\n{{\n");
+ foreach (var bodyModel in part.Models)
+ {
+ var meshes = bodyModel.Lods.Count > 0 ? bodyModel.Lods[0].Meshes : [];
+ if (meshes.Count == 0)
+ {
+ files.Add(null);
+ qc.Append("\tblank\n");
+ continue;
+ }
+ var stem = SafeName(Path.GetFileNameWithoutExtension(bodyModel.Name.Replace('\\', '/')));
+ var file = unique(stem.Length > 0 ? stem : SafeName(part.Name) is { Length: > 0 } partStem ? partStem : "body", ".smd");
+ WriteSmd(folder, file, stream => SmdFormat.WriteSkeletal(details.Model, meshes, stream, cancellationToken));
+ files.Add(file);
+ qc.Append(Invariant, $"\tstudio {Quote(file)}\n");
+ }
+ qc.Append("}\n\n");
+ references.Add(files);
+ }
+ qc.Append('\n');
+ return references;
+ }
+
+ ///
+ /// A $lod block (or $shadowlod) for each LOD after the first: models whose triangles change get their own SMD
+ /// (shared with the LOD before when the same), models with none are removed.
+ ///
+ private static void WriteLods(StringBuilder qc, SourceModelDetails details, List> references, string folder,
+ Func unique, CancellationToken cancellationToken)
+ {
+ var lods = details.BodyParts.SelectMany(p => p.Models).Select(m => m.Lods.Count).DefaultIfEmpty(0).Max();
+ // The file each model's LOD so far uses, to reuse it when the next LOD is the same.
+ var current = references.Select(files => files.ToList()).ToList();
+ var currentMeshes = details.BodyParts.Select(p => p.Models.Select(m => m.Lods.Count > 0 ? m.Lods[0].Meshes : []).ToList()).ToList();
+ for (var lod = 1; lod < lods; lod++)
+ {
+ var switchPoint = details.BodyParts.SelectMany(p => p.Models).Where(m => m.Lods.Count > lod).Select(m => m.Lods[lod].SwitchPoint).FirstOrDefault();
+ var block = new StringBuilder();
+ for (var p = 0; p < details.BodyParts.Count; p++)
+ {
+ for (var m = 0; m < details.BodyParts[p].Models.Count; m++)
+ {
+ var bodyModel = details.BodyParts[p].Models[m];
+ var reference = references[p][m];
+ if (reference is null || bodyModel.Lods.Count <= lod)
+ continue;
+ var meshes = bodyModel.Lods[lod].Meshes;
+ if (meshes.Count == 0)
+ {
+ block.Append(Invariant, $"\tremovemodel {Quote(reference)}\n");
+ continue;
+ }
+ if (!SameMeshes(meshes, currentMeshes[p][m]))
+ {
+ current[p][m] = unique($"{Path.GetFileNameWithoutExtension(reference)}_lod{lod}", ".smd");
+ currentMeshes[p][m] = meshes;
+ var file = current[p][m]!;
+ WriteSmd(folder, file, stream => SmdFormat.WriteSkeletal(details.Model, meshes, stream, cancellationToken));
+ }
+ if (!string.Equals(current[p][m], reference, StringComparison.OrdinalIgnoreCase))
+ block.Append(Invariant, $"\treplacemodel {Quote(reference)} {Quote(current[p][m]!)}\n");
+ }
+ }
+ qc.Append(switchPoint < 0 ? "$shadowlod\n" : $"$lod {Number(switchPoint)}\n").Append("{\n").Append(block).Append("}\n");
+ }
+ if (lods > 1)
+ qc.Append('\n');
+ }
+
+ private static bool SameMeshes(List a, List b) =>
+ a.Count == b.Count && a.Zip(b).All(pair => pair.First.Material == pair.Second.Material && pair.First.Triangles.SequenceEqual(pair.Second.Triangles));
+
+ /// $cdmaterials, and $texturegroup with the textures that differ between skins.
+ private static void WriteMaterials(StringBuilder qc, SourceModelDetails details)
+ {
+ foreach (var path in details.MaterialFolders)
+ qc.Append(Invariant, $"$cdmaterials {Quote(path)}\n");
+ if (details.SkinFamilies.Count > 1)
+ {
+ var first = details.SkinFamilies[0];
+ var columns = Enumerable.Range(0, first.Length)
+ .Where(c => details.SkinFamilies.Any(row => row[c] != first[c]))
+ .Where(c => details.SkinFamilies.All(row => row[c] >= 0 && row[c] < details.Textures.Count))
+ .ToList();
+ if (columns.Count > 0)
+ {
+ qc.Append("$texturegroup \"skinfamilies\"\n{\n");
+ foreach (var row in details.SkinFamilies)
+ qc.Append("\t{ ").AppendJoin(' ', columns.Select(c => Quote(details.Textures[row[c]]))).Append(" }\n");
+ qc.Append("}\n");
+ }
+ }
+ qc.Append('\n');
+ }
+
+ private static void WriteAttachments(StringBuilder qc, SourceModelDetails details)
+ {
+ var bones = details.Model.Bones;
+ foreach (var attachment in details.Attachments)
+ {
+ if (attachment.Bone < 0 || attachment.Bone >= bones.Count)
+ continue;
+ // The QC's "rotate" takes pitch, yaw and roll in degrees: the rotations about Y, Z and X.
+ var r = attachment.Rotation * (180 / MathF.PI);
+ qc.Append(Invariant, $"$attachment {Quote(attachment.Name)} {Quote(bones[attachment.Bone].Name)} {Vector(attachment.Position)}");
+ qc.Append(Invariant, $" rotate {Number(r.Y)} {Number(r.Z)} {Number(r.X)}\n");
+ }
+ if (details.Attachments.Count > 0)
+ qc.Append('\n');
+ }
+
+ /// The hitboxes, unless studiomdl made them itself (then a recompile makes them again), as Crowbar does.
+ private static void WriteHitboxes(StringBuilder qc, SourceModelDetails details)
+ {
+ if ((details.Flags & AutogeneratedHitboxes) != 0)
+ return;
+ var bones = details.Model.Bones;
+ foreach (var set in details.HitboxSets.Where(s => s.Boxes.Count > 0))
+ {
+ qc.Append(Invariant, $"$hboxset {Quote(set.Name)}\n");
+ foreach (var box in set.Boxes.Where(b => b.Bone >= 0 && b.Bone < bones.Count))
+ qc.Append(Invariant, $"$hbox {box.Group} {Quote(bones[box.Bone].Name)} {Vector(box.Min)} {Vector(box.Max)}\n");
+ qc.Append('\n');
+ }
+ }
+
+ /// One animation SMD per animation in "<model>_anims", and the $sequence blocks that use them.
+ private static void WriteAnimations(StringBuilder qc, SourceModelDetails details, SourceBytes? ani, string name, string folder,
+ CancellationToken cancellationToken)
+ {
+ if (details.Animations.Count == 0)
+ return;
+ var animFolder = name + "_anims";
+ Directory.CreateDirectory(Path.Combine(folder, animFolder));
+ var used = new HashSet(StringComparer.OrdinalIgnoreCase);
+ var paths = new List();
+ foreach (var animation in details.Animations)
+ {
+ var stem = SafeName(animation.Name.TrimStart('@'));
+ if (stem.Length == 0)
+ stem = "anim";
+ var file = stem + ".smd";
+ for (var n = 2; !used.Add(file); n++)
+ file = $"{stem}_{n}.smd";
+ var frames = SourceAnimation.Decode(details, animation, ani, cancellationToken);
+ WriteSmd(Path.Combine(folder, animFolder), file, stream => SmdFormat.WriteAnimation(details.Model.Bones, frames, stream, cancellationToken));
+ paths.Add($"{animFolder}\\{file}");
+ }
+
+ // $sectionframes before the animations: the frames a section holds, and the fewest frames an animation needs to be
+ // cut into sections (the shortest one that was).
+ var sectioned = details.Animations.Where(a => a.SectionFrames > 0).ToList();
+ if (sectioned.Count > 0)
+ qc.Append(Invariant, $"$sectionframes {sectioned[0].SectionFrames} {sectioned.Min(a => a.Frames)}\n\n");
+
+ var referenced = new HashSet(details.Sequences.SelectMany(s => s.Animations));
+ for (var a = 0; a < details.Animations.Count; a++)
+ {
+ if (!referenced.Contains(a))
+ qc.Append(Invariant, $"$animation {Quote(details.Animations[a].Name.TrimStart('@'))} {Quote(paths[a])} fps {Number(details.Animations[a].Fps)}\n");
+ }
+ foreach (var sequence in details.Sequences)
+ WriteSequence(qc, details, sequence, paths);
+ }
+
+ private static void WriteSequence(StringBuilder qc, SourceModelDetails details, SourceModelDetails.Sequence sequence, List paths)
+ {
+ var first = details.Animations[sequence.Animations[0]];
+ qc.Append(Invariant, $"$sequence {Quote(sequence.Name)} {{\n");
+ foreach (var animation in sequence.Animations)
+ qc.Append(Invariant, $"\t{Quote(paths[animation])}\n");
+ if (sequence.Activity.Length > 0)
+ qc.Append(Invariant, $"\tactivity {Quote(sequence.Activity)} {sequence.ActivityWeight}\n");
+ qc.Append(Invariant, $"\tfadein {Number(sequence.FadeIn)}\n");
+ qc.Append(Invariant, $"\tfadeout {Number(sequence.FadeOut)}\n");
+ qc.Append(Invariant, $"\tfps {Number(first.Fps)}\n");
+ foreach (var (flag, option) in SequenceFlags)
+ {
+ if ((sequence.Flags & flag) != 0)
+ qc.Append('\t').Append(option).Append('\n');
+ }
+ var motion = SourceAnimation.ReadMovements(details.Model.Mdl, first.Offset).Aggregate(0, (flags, m) => flags | m.Flags);
+ var motionNames = SourceAnimation.MotionFlags.Where(f => (motion & f.Flag) != 0).Select(f => f.Name).ToList();
+ if (motionNames.Count > 0)
+ qc.Append('\t').AppendJoin(' ', motionNames).Append('\n');
+ if (sequence.Animations.Count > 1)
+ {
+ qc.Append(Invariant, $"\tblendwidth {sequence.BlendWidth}\n");
+ for (var p = 0; p < 2; p++)
+ {
+ var index = sequence.Parameters[p];
+ if (index >= 0 && index < details.PoseParameters.Count)
+ qc.Append(Invariant, $"\tblend {Quote(details.PoseParameters[index].Name)} {Number(sequence.ParameterStart[p])} {Number(sequence.ParameterEnd[p])}\n");
+ }
+ }
+ foreach (var e in sequence.Events)
+ {
+ var frame = (int)MathF.Round(e.Cycle * Math.Max(first.Frames - 1, 0));
+ var id = (e.Type & NamedEvent) != 0 && e.Name.Length > 0 ? e.Name : e.Event.ToString(Invariant);
+ qc.Append(Invariant, $"\t{{ event {id} {frame} {Quote(e.Options)} }}\n");
+ }
+ qc.Append("}\n\n");
+ }
+
+ ///
+ /// The physics SMD and its $collisionmodel, or $collisionjoints when every solid is named after a bone (a ragdoll,
+ /// whose pieces are in their bones' space and are moved into the model's).
+ ///
+ private static void WritePhysics(StringBuilder qc, SourceModelDetails details, SourcePhysics physics, string file, string folder,
+ CancellationToken cancellationToken)
+ {
+ var bones = details.Model.Bones;
+ if (physics.Checksum != details.Checksum || physics.Solids.Count == 0)
+ {
+ qc.Append("// The .phy beside the model belongs to another build of it, so no collision model is written.\n\n");
+ return;
+ }
+ var boneIndex = new Dictionary(StringComparer.OrdinalIgnoreCase);
+ for (var i = 0; i < bones.Count; i++)
+ boneIndex.TryAdd(bones[i].Name, i);
+ var solidBlocks = physics.Blocks.Where(b => b.Name.Equals("solid", StringComparison.OrdinalIgnoreCase)).ToList();
+ string? SolidName(int solid) =>
+ solidBlocks.FirstOrDefault(b => b.Values.GetValueOrDefault("index") == solid.ToString(Invariant))?.Values.GetValueOrDefault("name");
+ var joints = Enumerable.Range(0, physics.Solids.Count).All(s => SolidName(s) is { } bone && boneIndex.ContainsKey(bone));
+ var edit = physics.Blocks.FirstOrDefault(b => b.Name.Equals("editparams", StringComparison.OrdinalIgnoreCase))?.Values
+ ?? new Dictionary();
+ var root = edit.GetValueOrDefault("rootname") is { Length: > 0 } rootName && boneIndex.TryGetValue(rootName, out var rootBone) ? rootBone : 0;
+ var world = WorldMatrices(bones);
+
+ var pieces = new List<(int Bone, IReadOnlyList Points, IReadOnlyList Triangles)>();
+ for (var s = 0; s < physics.Solids.Count; s++)
+ {
+ var bone = joints ? boneIndex[SolidName(s)!] : root;
+ foreach (var piece in physics.Solids[s].Pieces)
+ {
+ IReadOnlyList points = joints && bone < world.Length
+ ? piece.Points.Select(p => Vector3.Transform(p, world[bone])).ToList()
+ : piece.Points;
+ pieces.Add((bone, points, piece.Triangles));
+ }
+ }
+ WriteSmd(folder, file, stream => SmdFormat.WritePhysics(bones, pieces, stream, cancellationToken));
+
+ var firstSolid = solidBlocks.FirstOrDefault()?.Values ?? new Dictionary();
+ qc.Append(Invariant, $"{(joints ? "$collisionjoints" : "$collisionmodel")} {Quote(file)}\n{{\n");
+ var mass = edit.GetValueOrDefault("totalmass") ?? firstSolid.GetValueOrDefault("mass");
+ if (mass is not null)
+ qc.Append(Invariant, $"\t$mass {Value(mass)}\n");
+ foreach (var key in new[] { "inertia", "damping", "rotdamping" })
+ {
+ if (firstSolid.GetValueOrDefault(key) is { } value)
+ qc.Append(Invariant, $"\t${key} {Value(value)}\n");
+ }
+ if (edit.GetValueOrDefault("rootname") is { Length: > 0 } rootBoneName)
+ qc.Append(Invariant, $"\t$rootbone {Quote(rootBoneName)}\n");
+ if (!joints && (edit.GetValueOrDefault("concave") == "1" || pieces.Count > 1))
+ qc.Append("\t$concave\n");
+ if (pieces.Count > 20)
+ qc.Append(Invariant, $"\t$maxconvexpieces {pieces.Count}\n");
+ if (joints)
+ {
+ foreach (var constraint in physics.Blocks.Where(b => b.Name.Equals("ragdollconstraint", StringComparison.OrdinalIgnoreCase)))
+ {
+ if (!int.TryParse(constraint.Values.GetValueOrDefault("child"), NumberStyles.Integer, Invariant, out var child)
+ || SolidName(child) is not { } childBone)
+ continue;
+ foreach (var axis in new[] { "x", "y", "z" })
+ {
+ var min = Value(constraint.Values.GetValueOrDefault(axis + "min") ?? "0");
+ var max = Value(constraint.Values.GetValueOrDefault(axis + "max") ?? "0");
+ var friction = Value(constraint.Values.GetValueOrDefault(axis + "friction") ?? "0");
+ qc.Append(min == "0" && max == "0"
+ ? $"\t$jointconstrain {Quote(childBone)} {axis} fixed\n"
+ : $"\t$jointconstrain {Quote(childBone)} {axis} limit {min} {max} {friction}\n");
+ }
+ }
+ }
+ qc.Append("}\n\n");
+ }
+
+ /// Each bone's bind pose in model space (System.Numerics' row-vector matrices).
+ internal static Matrix4x4[] WorldMatrices(IReadOnlyList bones)
+ {
+ var world = new Matrix4x4[bones.Count];
+ for (var i = 0; i < bones.Count; i++)
+ {
+ var local = Matrix4x4.CreateFromQuaternion(SourceAnimation.FromEuler(bones[i].Rotation)) * Matrix4x4.CreateTranslation(bones[i].Position);
+ // Bones are stored parents first; a parent that comes later (a damaged file) is treated as none.
+ var parent = bones[i].Parent;
+ world[i] = parent >= 0 && parent < i ? local * world[parent] : local;
+ }
+ return world;
+ }
+
+ private static void WriteSmd(string folder, string file, Action write)
+ {
+ using var stream = new FileStream(Path.Combine(folder, file), FileMode.CreateNew, FileAccess.Write, FileShare.None, 1 << 16);
+ write(stream);
+ }
+
+ /// A file name made of the name's letters, digits and "-_." (others become "_").
+ internal static string SafeName(string name)
+ {
+ var chars = name.Select(c => char.IsLetterOrDigit(c) || c is '-' or '_' or '.' ? c : '_').ToArray();
+ return new string(chars).Trim('.');
+ }
+
+ private static string Quote(string text) => "\"" + text.Replace('"', '\'') + "\"";
+
+ /// A number with up to six decimals, never "-0".
+ private static string Number(float value)
+ {
+ var text = value.ToString("0.######", Invariant);
+ return text == "-0" ? "0" : text;
+ }
+
+ private static string Vector(Vector3 v) => $"{Number(v.X)} {Number(v.Y)} {Number(v.Z)}";
+
+ /// A number from the .phy's text as the QC writes it.
+ private static string Value(string text) =>
+ float.TryParse(text, NumberStyles.Float, Invariant, out var value) ? Number(value) : "0";
+
+ private static string Indent(string text) => string.Join('\n', text.Split('\n').Select(line => "\t" + line.Trim()));
+}
diff --git a/src/Filee.Engines/Models3D/SourceModel.cs b/src/Filee.Engines/Models3D/SourceModel.cs
new file mode 100644
index 000000000..ee0fa60bc
--- /dev/null
+++ b/src/Filee.Engines/Models3D/SourceModel.cs
@@ -0,0 +1,555 @@
+// Source engine models (.mdl, versions 44 to 49: Garry's Mod, Counter-Strike: Source, Team Fortress 2, Half-Life 2), read
+// by Filee with the two files studiomdl writes beside them: the vertices (.vvd) and the triangles (.dx90.vtx, else .vtx,
+// .dx80.vtx or .sw.vtx). Layouts (little endian) as in Valve's studio.h / optimize.h and the readers of SourceIO and
+// Crowbar:
+//
+// .mdl studiohdr_t (408 bytes): bones (216 bytes each: name, parent, position, rotation), textures (64 bytes each:
+// name), texture folders ($cdmaterials), the skin table (shorts: family × reference), body parts (16 bytes: name,
+// models) → models (148 bytes: name, meshes, first vertex) → meshes (116 bytes: material, vertex offset)
+// .vvd "IDSV", version 4, the checksum of its .mdl, vertices per LOD, fixups (which runs of vertices LOD 0 uses);
+// 48 bytes a vertex: three bone weights, three bones, bone count, position, normal, UV
+// .vtx version 7: body parts → models → LODs → meshes (9 bytes) → strip groups (25 bytes; 33 in later branches)
+// → their vertices (9 bytes: the index of the mesh's vertex) and triangle lists (or strips) of u16
+//
+// The model is read in its bind pose with the default body groups (the first model of each body part), the first skin
+// and LOD 0; the rest (every body group and LOD, animations, physics) is read for decompiling (SourceDecompiler).
+// Source is Z up, one unit an inch, triangles clockwise, UVs from the top left: Filee turns that into its own
+// conventions (Y up, millimetres, counter-clockwise, bottom left).
+// The textures are found as Garry's Mod finds them: the model's material (a .vmt) in a "materials" folder above the
+// model, its $basetexture (a .vtf) decoded by the VTF reader.
+
+using System.Numerics;
+using Filee.Engines.Magick;
+using ImageMagick;
+
+namespace Filee.Engines.Models3D;
+
+/// A Source engine model as read from its .mdl, .vvd and .vtx.
+internal sealed partial class SourceModel
+{
+ /// One inch, Source's unit, in Filee's millimetres.
+ public const float MillimetresPerUnit = 25.4f;
+
+ internal const int MaxBones = 1024;
+ internal const int MaxTextures = 4096;
+ internal const int MaxBodyParts = 512;
+ internal const int MaxMeshes = 16384;
+ internal const int MaxVertices = 4_000_000;
+
+ /// A bone in its bind pose, relative to its parent (Source's axes and units).
+ internal readonly record struct Bone(string Name, int Parent, Vector3 Position, Vector3 Rotation);
+
+ /// A vertex in Source's axes and units, UV from the top left, with up to three bone weights.
+ internal readonly record struct Vertex(Vector3 Position, Vector3 Normal, Vector2 Uv, int BoneCount,
+ int Bone0, int Bone1, int Bone2, float Weight0, float Weight1, float Weight2);
+
+ /// One mesh: its part name, material and triangles (indices into , Source's winding).
+ internal sealed record Mesh(string Part, string Material, List Triangles);
+
+ public string Name { get; private init; } = "";
+ public List Bones { get; } = [];
+ public List Vertices { get; } = [];
+ public List Meshes { get; } = [];
+
+ /// The folders the model's materials are looked up in ($cdmaterials, "/" separated, without the end slash).
+ public List MaterialFolders { get; } = [];
+
+ /// The material of every skin reference in the first skin family (what a mesh's material number selects).
+ public List SkinMaterials { get; private set; } = [];
+
+ /// The .mdl, its file name and folder; the .vtx and its layout, kept for decompiling.
+ internal SourceBytes Mdl { get; private init; } = new([], "");
+ internal SourceBytes Vtx { get; private set; } = new([], "");
+ internal bool ExtendedVtx { get; private set; }
+ internal string MdlPath { get; private init; } = "";
+
+ /// Reads the model at with its .vvd and .vtx.
+ public static SourceModel Load(string mdlPath, CancellationToken cancellationToken)
+ {
+ var mdl = SourceBytes.Read(mdlPath);
+ var basePath = Path.Combine(Path.GetDirectoryName(Path.GetFullPath(mdlPath))!, Path.GetFileNameWithoutExtension(mdlPath));
+ var vvdPath = Companion(basePath, [".vvd"])
+ ?? throw new FileNotFoundException($"{Path.GetFileName(mdlPath)} needs its vertex file {Path.GetFileName(basePath)}.vvd next to it.");
+ var vtxPath = Companion(basePath, [".dx90.vtx", ".vtx", ".dx80.vtx", ".sw.vtx"])
+ ?? throw new FileNotFoundException($"{Path.GetFileName(mdlPath)} needs its triangle file {Path.GetFileName(basePath)}.dx90.vtx next to it.");
+ var vvd = SourceBytes.Read(vvdPath);
+ var vtx = SourceBytes.Read(vtxPath);
+
+ if (!mdl.Data.AsSpan().StartsWith("IDST"u8))
+ throw new InvalidDataException($"{mdl.File} is not a Source engine model (IDST).");
+ var version = mdl.I32(4);
+ if (version is < 44 or > 49)
+ throw new InvalidDataException($"{mdl.File} is a model of version {version}; Filee reads Source engine models of versions 44 to 49.");
+ var checksum = mdl.I32(8);
+ var model = new SourceModel { Name = mdl.String(12, 64), Mdl = mdl, MdlPath = Path.GetFullPath(mdlPath) };
+
+ ReadBones(mdl, model);
+ var textures = ReadTextures(mdl, model);
+ model.SkinMaterials = textures;
+ ReadVertices(vvd, checksum, model);
+ cancellationToken.ThrowIfCancellationRequested();
+ if (!vtx.Data.AsSpan().StartsWith((ReadOnlySpan)[7, 0, 0, 0]))
+ throw new InvalidDataException($"{vtx.File} is not a triangle file of version 7.");
+ if (vtx.I32(16) != checksum)
+ throw new InvalidDataException($"{vtx.File} belongs to another build of {mdl.File}: its checksum differs.");
+ // Strip groups and strips grew by 8 bytes in later branches (CS:GO); the layout whose indices all fit is used.
+ model.Vtx = vtx;
+ try
+ {
+ ReadMeshes(mdl, vtx, textures, model, extended: false, cancellationToken);
+ }
+ catch (InvalidDataException)
+ {
+ model.Meshes.Clear();
+ model.ExtendedVtx = true;
+ ReadMeshes(mdl, vtx, textures, model, extended: true, cancellationToken);
+ }
+ if (model.Meshes.Count == 0)
+ throw new InvalidDataException($"{mdl.File} has no triangles in its default body groups.");
+ return model;
+ }
+
+ /// The first of " + suffix" that exists, letter case aside.
+ internal static string? Companion(string basePath, string[] suffixes)
+ {
+ var folder = Path.GetDirectoryName(basePath)!;
+ var name = Path.GetFileName(basePath);
+ var files = Directory.EnumerateFiles(folder).ToList();
+ foreach (var suffix in suffixes)
+ {
+ var match = files.FirstOrDefault(f => string.Equals(Path.GetFileName(f), name + suffix, StringComparison.OrdinalIgnoreCase));
+ if (match is not null)
+ return match;
+ }
+ return null;
+ }
+
+ private static void ReadBones(SourceBytes mdl, SourceModel model)
+ {
+ var count = mdl.Count(156, MaxBones, "bones");
+ var start = mdl.I32(160);
+ for (var i = 0; i < count; i++)
+ {
+ var at = start + i * 216;
+ var parent = mdl.I32(at + 4);
+ model.Bones.Add(new Bone(mdl.String(at + mdl.I32(at)), parent is >= -1 && parent < count ? parent : -1,
+ mdl.Vector(at + 32), mdl.Vector(at + 60)));
+ }
+ }
+
+ /// The texture name of every skin reference of the first skin family.
+ private static List ReadTextures(SourceBytes mdl, SourceModel model)
+ {
+ var count = mdl.Count(204, MaxTextures, "textures");
+ var start = mdl.I32(208);
+ var names = new List(count);
+ for (var i = 0; i < count; i++)
+ {
+ var at = start + i * 64;
+ names.Add(mdl.String(at + mdl.I32(at)).Replace('\\', '/'));
+ }
+
+ var folders = mdl.Count(212, 256, "texture folders");
+ var folderTable = mdl.I32(216);
+ for (var i = 0; i < folders; i++)
+ model.MaterialFolders.Add(mdl.String(mdl.I32(folderTable + i * 4)).Replace('\\', '/').Trim('/'));
+
+ // The skin table's first row maps each mesh's material reference to a texture.
+ var references = mdl.Count(220, MaxTextures, "skin references");
+ var families = mdl.Count(224, MaxTextures, "skins");
+ var skinTable = mdl.I32(228);
+ var skin = new List(references);
+ for (var i = 0; i < references; i++)
+ {
+ var texture = families > 0 ? mdl.I16(skinTable + i * 2) : i;
+ skin.Add(texture >= 0 && texture < names.Count ? names[texture] : i < names.Count ? names[i] : "");
+ }
+ return skin.Count > 0 ? skin : names;
+ }
+
+ private static void ReadVertices(SourceBytes vvd, int checksum, SourceModel model)
+ {
+ if (!vvd.Data.AsSpan().StartsWith("IDSV"u8))
+ throw new InvalidDataException($"{vvd.File} is not a Source engine vertex file (IDSV).");
+ if (vvd.I32(8) != checksum)
+ throw new InvalidDataException($"{vvd.File} belongs to another build of the model: its checksum differs.");
+ var lod0 = vvd.Count(16, MaxVertices, "vertices");
+ var fixups = vvd.Count(48, MaxVertices, "fixups");
+ var fixupTable = vvd.I32(52);
+ var vertexData = vvd.I32(56);
+ // The tables are checked against the file before a single vertex is read or added.
+ if (!vvd.Fits(vertexData, lod0, 48) || !vvd.Fits(fixupTable, fixups, 12))
+ throw vvd.Damaged("the vertex table");
+
+ void Add(int index)
+ {
+ var at = vertexData + index * 48;
+ model.Vertices.Add(new Vertex(vvd.Vector(at + 16), vvd.Vector(at + 28), new Vector2(vvd.F32(at + 40), vvd.F32(at + 44)),
+ Math.Clamp((int)vvd.Byte(at + 15), 1, 3), vvd.Byte(at + 12), vvd.Byte(at + 13), vvd.Byte(at + 14),
+ vvd.F32(at), vvd.F32(at + 4), vvd.F32(at + 8)));
+ }
+
+ if (fixups == 0)
+ {
+ for (var i = 0; i < lod0; i++)
+ Add(i);
+ return;
+ }
+ // With fixups, LOD 0 is every run of vertices whose LOD is 0 or above, in order.
+ for (var f = 0; f < fixups; f++)
+ {
+ var at = fixupTable + f * 12;
+ var lod = vvd.I32(at);
+ var source = vvd.I32(at + 4);
+ var length = vvd.I32(at + 8);
+ if (lod < 0)
+ continue;
+ if (source < 0 || length < 0 || model.Vertices.Count + length > MaxVertices || !vvd.Fits(vertexData, (long)source + length, 48))
+ throw vvd.Damaged("a vertex fixup");
+ for (var i = 0; i < length; i++)
+ Add(source + i);
+ }
+ }
+
+ private static void ReadMeshes(SourceBytes mdl, SourceBytes vtx, List materials, SourceModel model, bool extended, CancellationToken cancellationToken)
+ {
+ var bodyParts = Math.Min(mdl.Count(232, MaxBodyParts, "body parts"), vtx.Count(28, MaxBodyParts, "body parts"));
+ var mdlBodyParts = mdl.I32(236);
+ var vtxBodyParts = vtx.I32(32);
+ for (var b = 0; b < bodyParts; b++)
+ {
+ var mdlPart = mdlBodyParts + b * 16;
+ var vtxPart = vtxBodyParts + b * 8;
+ if (mdl.I32(mdlPart + 4) <= 0 || vtx.I32(vtxPart) <= 0)
+ continue;
+ // The default body group: the first model of the part, at LOD 0.
+ var mdlModel = mdlPart + mdl.I32(mdlPart + 12);
+ var vtxModel = vtxPart + vtx.I32(vtxPart + 4);
+ if (vtx.I32(vtxModel) <= 0)
+ continue;
+ var lod = vtxModel + vtx.I32(vtxModel + 4);
+ model.Meshes.AddRange(ReadLodMeshes(mdl, vtx, mdlModel, lod, materials, model.Vertices.Count, extended, cancellationToken));
+ }
+ }
+
+ ///
+ /// The meshes of one model at one LOD: the model's meshes in the .mdl (at ) with their
+ /// triangles from the .vtx LOD at , as indices into the LOD 0 vertices (every LOD's vertex
+ /// numbers count from the same first vertex of each mesh).
+ ///
+ internal static List ReadLodMeshes(SourceBytes mdl, SourceBytes vtx, int mdlModel, int vtxLod, IReadOnlyList materials,
+ int vertexCount, bool extended, CancellationToken cancellationToken)
+ {
+ var stripGroupSize = extended ? 33 : 25;
+ var stripSize = extended ? 35 : 27;
+ var modelName = mdl.String(mdlModel, 64);
+ var firstVertex = mdl.I32(mdlModel + 84) / 48;
+ var meshes = Math.Min(mdl.Count(mdlModel + 72, MaxMeshes, "meshes"), vtx.Count(vtxLod, MaxMeshes, "meshes"));
+ var mdlMeshes = mdlModel + mdl.I32(mdlModel + 76);
+ var vtxMeshes = vtxLod + vtx.I32(vtxLod + 4);
+ var result = new List();
+
+ for (var m = 0; m < meshes; m++)
+ {
+ cancellationToken.ThrowIfCancellationRequested();
+ var mdlMesh = mdlMeshes + m * 116;
+ var material = mdl.I32(mdlMesh);
+ var meshFirst = firstVertex + mdl.I32(mdlMesh + 12);
+ var vtxMesh = vtxMeshes + m * 9;
+ var groups = vtx.Count(vtxMesh, MaxMeshes, "strip groups");
+ var groupTable = vtxMesh + vtx.I32(vtxMesh + 4);
+ var triangles = new List();
+ for (var g = 0; g < groups; g++)
+ {
+ var group = groupTable + g * stripGroupSize;
+ var groupVertices = vtx.Count(group, MaxVertices, "vertices");
+ var vertexTable = group + vtx.I32(group + 4);
+ var indexCount = vtx.Count(group + 8, MaxVertices * 3, "indices");
+ var indexTable = group + vtx.I32(group + 12);
+ var strips = vtx.Count(group + 16, MaxMeshes, "strips");
+ var stripTable = group + vtx.I32(group + 20);
+
+ int MeshVertex(int index)
+ {
+ if ((uint)index >= (uint)groupVertices)
+ throw vtx.Damaged("a triangle");
+ var vertex = meshFirst + vtx.U16(vertexTable + index * 9 + 4);
+ if ((uint)vertex >= (uint)vertexCount)
+ throw vtx.Damaged("a triangle");
+ return vertex;
+ }
+
+ var anyStrip = false;
+ for (var s = 0; s < strips; s++)
+ {
+ var strip = stripTable + s * stripSize;
+ if ((vtx.Byte(strip + 18) & 0x02) == 0)
+ continue;
+ // A triangle strip: every next index makes a triangle with the two before it, alternating.
+ anyStrip = true;
+ var first = vtx.I32(strip + 4);
+ var count = vtx.I32(strip);
+ if (first < 0 || count < 0 || first + count > indexCount)
+ throw vtx.Damaged("a triangle strip");
+ for (var i = 2; i < count; i++)
+ {
+ var a = MeshVertex(vtx.U16(indexTable + (first + i - 2) * 2));
+ var c = MeshVertex(vtx.U16(indexTable + (first + i - 1) * 2));
+ var d = MeshVertex(vtx.U16(indexTable + (first + i) * 2));
+ if (a == c || c == d || a == d)
+ continue;
+ if (i % 2 == 0)
+ triangles.AddRange([a, c, d]);
+ else
+ triangles.AddRange([c, a, d]);
+ }
+ }
+ if (anyStrip)
+ continue;
+ if (indexCount % 3 != 0)
+ throw vtx.Damaged("a triangle list");
+ for (var i = 0; i < indexCount; i++)
+ triangles.Add(MeshVertex(vtx.U16(indexTable + i * 2)));
+ }
+ if (triangles.Count == 0)
+ continue;
+ var name = material >= 0 && material < materials.Count ? materials[material] : $"material{material}";
+ result.Add(new Mesh(modelName, name, triangles));
+ }
+ return result;
+ }
+
+ ///
+ /// The model in Filee's conventions: one part per mesh, its textures looked up in the "materials" folder above the
+ /// model (none when there is no such folder or the game's own textures are meant).
+ ///
+ public Model3D ToModel3D(string mdlPath, CancellationToken cancellationToken)
+ {
+ var result = new Model3D();
+ var textures = new SourceMaterials(mdlPath, MaterialFolders);
+ var materialIndex = new Dictionary(StringComparer.OrdinalIgnoreCase);
+ foreach (var mesh in Meshes)
+ {
+ cancellationToken.ThrowIfCancellationRequested();
+ if (!materialIndex.TryGetValue(mesh.Material, out var index))
+ {
+ index = result.Materials.Count;
+ materialIndex[mesh.Material] = index;
+ var (texture, doubleSided) = textures.Find(mesh.Material);
+ result.Materials.Add(new Material3D
+ {
+ Name = Path.GetFileName(mesh.Material),
+ BaseColorTexture = texture,
+ DoubleSided = doubleSided,
+ });
+ }
+
+ var part = new MeshPart { Name = mesh.Part, Material = index, Normals = [], TexCoords = [] };
+ var local = new Dictionary();
+ int Local(int vertex)
+ {
+ if (local.TryGetValue(vertex, out var at))
+ return at;
+ var v = Vertices[vertex];
+ at = part.AddVertex(MeshMath.ToYUp(v.Position) * MillimetresPerUnit, MeshMath.ToYUp(v.Normal), new Vector2(v.Uv.X, 1 - v.Uv.Y));
+ local[vertex] = at;
+ return at;
+ }
+ // Source's triangles are clockwise: reversed, they face the same way counter-clockwise.
+ for (var t = 0; t + 2 < mesh.Triangles.Count; t += 3)
+ part.AddTriangle(Local(mesh.Triangles[t + 2]), Local(mesh.Triangles[t + 1]), Local(mesh.Triangles[t]));
+ result.Parts.Add(part);
+ }
+ return result;
+ }
+}
+
+///
+/// Finds a Source model's textures as Garry's Mod does: "materials/<texture folder>/<texture>.vmt" beside
+/// the "models" folder the model is in, then that material's $basetexture as a .vtf. Only .vmt and .vtf files below that
+/// "materials" folder are read, never through "..".
+///
+internal sealed class SourceMaterials
+{
+ /// How far above the model its "models" folder is looked for (models/a/b/c/model.mdl is four).
+ private const int MaxLevels = 10;
+
+ private readonly string? _root;
+ private readonly IReadOnlyList _folders;
+ private readonly Dictionary _textures = new(StringComparer.OrdinalIgnoreCase);
+
+ public SourceMaterials(string mdlPath, IReadOnlyList folders)
+ {
+ _folders = folders.Count > 0 ? folders : [""];
+ // Only the game's or addon's own layout: a "models" folder above the model, with "materials" beside it.
+ var folder = new DirectoryInfo(Path.GetDirectoryName(Path.GetFullPath(mdlPath))!);
+ for (var level = 0; level < MaxLevels && folder.Parent is { } parent; level++, folder = parent)
+ {
+ if (!folder.Name.Equals("models", StringComparison.OrdinalIgnoreCase))
+ continue;
+ try
+ {
+ _root = parent.EnumerateDirectories()
+ .FirstOrDefault(d => d.Name.Equals("materials", StringComparison.OrdinalIgnoreCase))?.FullName;
+ }
+ catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
+ {
+ // a folder Filee may not list: no textures
+ }
+ break;
+ }
+ }
+
+ /// The texture of material , and whether the material is drawn from both sides.
+ public (Texture3D? Texture, bool DoubleSided) Find(string name)
+ {
+ if (_root is null)
+ return (null, false);
+ foreach (var folder in _folders)
+ {
+ var relative = folder.Length > 0 ? folder + "/" + name : name;
+ if (Resolve(relative + ".vmt") is { } vmt)
+ {
+ var material = Vmt.Read(vmt, path => Resolve(path.StartsWith("materials/", StringComparison.OrdinalIgnoreCase) ? path[10..] : path));
+ var texture = material.BaseTexture is { } baseTexture ? Texture(StripExtension(baseTexture) + ".vtf") : null;
+ return (texture, material.NoCull);
+ }
+ if (Resolve(relative + ".vtf") is not null)
+ return (Texture(relative + ".vtf"), false);
+ }
+ return (null, false);
+ }
+
+ private static string StripExtension(string path) =>
+ path.EndsWith(".vtf", StringComparison.OrdinalIgnoreCase) ? path[..^4] : path;
+
+ /// The decoded texture at (below "materials"), as PNG, once per file.
+ private Texture3D? Texture(string relative)
+ {
+ if (_textures.TryGetValue(relative, out var known))
+ return known;
+ Texture3D? texture = null;
+ if (Resolve(relative) is { } path)
+ {
+ try
+ {
+ using var frames = Vtf.Read(path);
+ if (frames.Count > 0)
+ texture = new Texture3D(frames[0].ToByteArray(MagickFormat.Png), "png", Path.GetFileNameWithoutExtension(path));
+ }
+ catch (Exception ex) when (ex is InvalidDataException or MagickException or IOException)
+ {
+ // A texture Filee can't read leaves the material plain.
+ }
+ }
+ _textures[relative] = texture;
+ return texture;
+ }
+
+ ///
+ /// The file at below "materials" ("/" or "\" separated), letter case aside (Source's file
+ /// system ignores it), or null.
+ ///
+ private string? Resolve(string relative)
+ {
+ var parts = relative.Replace('\\', '/').Split('/', StringSplitOptions.RemoveEmptyEntries);
+ if (parts.Length == 0 || parts.Any(p => p is "." or ".."))
+ return null;
+ var current = _root!;
+ for (var i = 0; i < parts.Length; i++)
+ {
+ var exact = Path.Combine(current, parts[i]);
+ var last = i == parts.Length - 1;
+ if (last ? File.Exists(exact) : Directory.Exists(exact))
+ {
+ current = exact;
+ continue;
+ }
+ var entries = last ? Directory.EnumerateFiles(current) : Directory.EnumerateDirectories(current);
+ var match = entries.FirstOrDefault(e => string.Equals(Path.GetFileName(e), parts[i], StringComparison.OrdinalIgnoreCase));
+ if (match is null)
+ return null;
+ current = match;
+ }
+ return current;
+ }
+}
+
+/// The few things Filee reads from a Valve material (.vmt, KeyValues text): $basetexture and $nocull.
+internal static class Vmt
+{
+ internal sealed record Material(string? BaseTexture, bool NoCull);
+
+ ///
+ /// Reads ; a "patch" material takes what it doesn't set from the material it includes
+ /// (found through , one level).
+ ///
+ public static Material Read(string path, Func resolve)
+ {
+ var tokens = Tokens(File.ReadAllText(path));
+ var material = Parse(tokens);
+ if (tokens.Count > 0 && tokens[0].Equals("patch", StringComparison.OrdinalIgnoreCase)
+ && Value(tokens, "include") is { } include && include.EndsWith(".vmt", StringComparison.OrdinalIgnoreCase)
+ && resolve(include.Replace('\\', '/')) is { } included && included != path)
+ {
+ var parent = Parse(Tokens(File.ReadAllText(included)));
+ material = new Material(material.BaseTexture ?? parent.BaseTexture, material.NoCull || parent.NoCull);
+ }
+ return material;
+ }
+
+ private static Material Parse(List tokens) =>
+ new(Value(tokens, "$basetexture"), Value(tokens, "$nocull") is "1");
+
+ /// The value after the first key , at any depth.
+ private static string? Value(List tokens, string key)
+ {
+ for (var i = 0; i + 1 < tokens.Count; i++)
+ {
+ if (tokens[i].Equals(key, StringComparison.OrdinalIgnoreCase) && tokens[i + 1] is not ("{" or "}"))
+ return tokens[i + 1];
+ }
+ return null;
+ }
+
+ /// KeyValues tokens: quoted or bare words and braces; "//" comments are skipped.
+ internal static List Tokens(string text)
+ {
+ var tokens = new List();
+ var i = 0;
+ while (i < text.Length)
+ {
+ var c = text[i];
+ if (char.IsWhiteSpace(c))
+ {
+ i++;
+ }
+ else if (c == '/' && i + 1 < text.Length && text[i + 1] == '/')
+ {
+ while (i < text.Length && text[i] != '\n')
+ i++;
+ }
+ else if (c is '{' or '}')
+ {
+ tokens.Add(c.ToString());
+ i++;
+ }
+ else if (c == '"')
+ {
+ var end = text.IndexOf('"', i + 1);
+ if (end < 0)
+ end = text.Length;
+ tokens.Add(text[(i + 1)..end]);
+ i = end + 1;
+ }
+ else
+ {
+ var start = i;
+ while (i < text.Length && !char.IsWhiteSpace(text[i]) && text[i] is not ('{' or '}' or '"'))
+ i++;
+ tokens.Add(text[start..i]);
+ }
+ }
+ return tokens;
+ }
+}
diff --git a/src/Filee.Engines/Models3D/SourceModelDetails.cs b/src/Filee.Engines/Models3D/SourceModelDetails.cs
new file mode 100644
index 000000000..430e9e440
--- /dev/null
+++ b/src/Filee.Engines/Models3D/SourceModelDetails.cs
@@ -0,0 +1,301 @@
+// What a Source engine model (.mdl) holds besides its default mesh, read for decompiling it into a QC and its SMDs
+// (#1526). Layouts (little endian) as in Valve's studio.h, learned from it and from Crowbar's and SourceIO's readers:
+//
+// studiohdr_t eye position 80, illumination position 92, hull 104/116, view box 128/140, flags 152,
+// hitbox sets 172, animations 180, sequences 188, skin table 220, body parts 232, attachments 240,
+// pose parameters 300, surface property 308, key values 312, mass 328, contents 332, include
+// models 336, animation blocks 348, the directional light dot 376 (one byte)
+// mstudiobone_t 216 bytes; position scale 72 and rotation scale 84 turn compressed animation values into units
+// mstudiobodyparts_t 16 bytes → mstudiomodel_t 148 bytes (name, meshes, first vertex); the .vtx has a matching
+// model → LODs (12 bytes: meshes, offset, switch point) → meshes
+// mstudioattachment_t 92 bytes: name, flags, bone, a 3×4 matrix relative to the bone
+// mstudiohitboxset_t 12 bytes → mstudiobbox_t 68 bytes: bone, group, min, max, name
+// mstudioanimdesc_t 100 bytes: name, fps, flags, frames, movements, animation block and offset, sections
+// mstudioseqdesc_t 212 bytes: name, activity, flags, weight, events (80 bytes each), blend grid of animations,
+// blend parameters, fade times
+// mstudioposeparamdesc_t 20 bytes: name, flags, start, end, loop
+//
+// Every count is checked against a limit and every offset against the file before it is used.
+
+using System.Numerics;
+
+namespace Filee.Engines.Models3D;
+
+/// The parts of a Source model that a decompiled QC describes.
+internal sealed class SourceModelDetails
+{
+ private const int MaxLods = 8;
+ private const int MaxAttachments = 4096;
+ private const int MaxHitboxes = 16384;
+ private const int MaxAnimations = 16384;
+ private const int MaxEvents = 4096;
+ private const int MaxBlends = 4096;
+
+ internal sealed record BodyModel(string Name, List Lods);
+ internal sealed record BodyPart(string Name, List Models);
+ /// One LOD of a body model: its switch point (−1 for the shadow LOD) and meshes (empty when removed).
+ internal sealed record ModelLod(float SwitchPoint, List Meshes);
+ /// An attachment: its bone, and position and rotation (radians, Source's Euler order) relative to it.
+ internal sealed record Attachment(string Name, int Bone, Vector3 Position, Vector3 Rotation);
+ internal sealed record Hitbox(int Bone, int Group, Vector3 Min, Vector3 Max);
+ internal sealed record HitboxSet(string Name, List Boxes);
+ internal sealed record PoseParameter(string Name, int Flags, float Start, float End, float Loop);
+ /// An animation's description; its frames are decoded by .
+ internal sealed record Animation(int Offset, string Name, float Fps, int Flags, int Frames, int SectionFrames);
+ internal sealed record AnimationEvent(float Cycle, int Event, int Type, string Options, string Name);
+ internal sealed record Sequence(string Name, string Activity, int ActivityWeight, int Flags, List Events,
+ int BlendWidth, List Animations, int[] Parameters, float[] ParameterStart, float[] ParameterEnd, float FadeIn, float FadeOut);
+
+ public required SourceModel Model { get; init; }
+ public int Version { get; private init; }
+ public int Checksum { get; private init; }
+ public int Flags { get; private init; }
+ public Vector3 EyePosition { get; private init; }
+ public Vector3 IlluminationPosition { get; private init; }
+ public Vector3 HullMin { get; private init; }
+ public Vector3 HullMax { get; private init; }
+ public Vector3 ViewMin { get; private init; }
+ public Vector3 ViewMax { get; private init; }
+ public float Mass { get; private init; }
+ public int Contents { get; private init; }
+ public byte DirectionalLightDot { get; private init; }
+ public string SurfaceProperty { get; private init; } = "";
+ /// The model's key values ($keyvalues), as the text inside "mdlkeyvalue { … }".
+ public string KeyValues { get; private init; } = "";
+
+ /// Every texture name, and the texture number of each skin reference in each skin family.
+ public List Textures { get; } = [];
+ public List SkinFamilies { get; } = [];
+ public List MaterialFolders { get; } = [];
+ public List BodyParts { get; } = [];
+ public List Attachments { get; } = [];
+ public List HitboxSets { get; } = [];
+ public List PoseParameters { get; } = [];
+ public List IncludeModels { get; } = [];
+ public List Animations { get; } = [];
+ public List Sequences { get; } = [];
+ /// The scale of each bone's compressed animated position and rotation values.
+ public List<(Vector3 Position, Vector3 Rotation)> BoneScales { get; } = [];
+ /// Where each animation block lies in the model's .ani file (block 0 is the .mdl itself).
+ public List<(int Start, int End)> AnimationBlocks { get; } = [];
+
+ /// Reads the details of , which was loaded from its .mdl, .vvd and .vtx.
+ public static SourceModelDetails Read(SourceModel model, CancellationToken cancellationToken)
+ {
+ var mdl = model.Mdl;
+ var details = new SourceModelDetails
+ {
+ Model = model,
+ Version = mdl.I32(4),
+ Checksum = mdl.I32(8),
+ EyePosition = mdl.Vector(80),
+ IlluminationPosition = mdl.Vector(92),
+ HullMin = mdl.Vector(104),
+ HullMax = mdl.Vector(116),
+ ViewMin = mdl.Vector(128),
+ ViewMax = mdl.Vector(140),
+ Flags = mdl.I32(152),
+ SurfaceProperty = mdl.I32(308) == 0 ? "" : mdl.String(mdl.I32(308)),
+ KeyValues = ReadKeyValues(mdl),
+ Mass = mdl.F32(328),
+ Contents = mdl.I32(332),
+ DirectionalLightDot = mdl.Byte(376),
+ };
+ details.ReadBoneScales(mdl);
+ details.ReadTextures(mdl);
+ details.ReadBodyParts(cancellationToken);
+ details.ReadAttachments(mdl);
+ details.ReadHitboxes(mdl);
+ details.ReadPoseParameters(mdl);
+ details.ReadIncludeModels(mdl);
+ details.ReadAnimations(mdl);
+ details.ReadSequences(mdl, cancellationToken);
+ return details;
+ }
+
+ /// The text inside "mdlkeyvalue { … }", the form $keyvalues takes in a QC.
+ private static string ReadKeyValues(SourceBytes mdl)
+ {
+ var size = mdl.I32(316);
+ if (size <= 0 || !mdl.Fits(mdl.I32(312), size, 1))
+ return "";
+ var text = mdl.String(mdl.I32(312), size);
+ var open = text.IndexOf('{');
+ var close = text.LastIndexOf('}');
+ return open >= 0 && close > open ? text[(open + 1)..close].Trim() : text.Trim();
+ }
+
+ private void ReadBoneScales(SourceBytes mdl)
+ {
+ var start = mdl.I32(160);
+ for (var i = 0; i < Model.Bones.Count; i++)
+ {
+ var at = start + i * 216;
+ BoneScales.Add((mdl.Vector(at + 72), mdl.Vector(at + 84)));
+ }
+ }
+
+ private void ReadTextures(SourceBytes mdl)
+ {
+ var count = mdl.Count(204, SourceModel.MaxTextures, "textures");
+ var start = mdl.I32(208);
+ for (var i = 0; i < count; i++)
+ Textures.Add(mdl.String(start + i * 64 + mdl.I32(start + i * 64)).Replace('\\', '/'));
+ var folders = mdl.Count(212, 256, "texture folders");
+ var folderTable = mdl.I32(216);
+ for (var i = 0; i < folders; i++)
+ MaterialFolders.Add(mdl.String(mdl.I32(folderTable + i * 4)));
+
+ var references = mdl.Count(220, SourceModel.MaxTextures, "skin references");
+ var families = mdl.Count(224, SourceModel.MaxTextures, "skins");
+ var table = mdl.I32(228);
+ if (!mdl.Fits(table, (long)references * families, 2))
+ throw mdl.Damaged("the skin table");
+ for (var f = 0; f < families; f++)
+ {
+ var row = new int[references];
+ for (var r = 0; r < references; r++)
+ row[r] = mdl.I16(table + (f * references + r) * 2);
+ SkinFamilies.Add(row);
+ }
+ }
+
+ /// Every model of every body part, at every LOD the .vtx has.
+ private void ReadBodyParts(CancellationToken cancellationToken)
+ {
+ var (mdl, vtx) = (Model.Mdl, Model.Vtx);
+ var parts = Math.Min(mdl.Count(232, SourceModel.MaxBodyParts, "body parts"), vtx.Count(28, SourceModel.MaxBodyParts, "body parts"));
+ var mdlParts = mdl.I32(236);
+ var vtxParts = vtx.I32(32);
+ for (var b = 0; b < parts; b++)
+ {
+ var mdlPart = mdlParts + b * 16;
+ var vtxPart = vtxParts + b * 8;
+ var part = new BodyPart(mdl.RelativeString(mdlPart, mdlPart), []);
+ var models = Math.Min(mdl.Count(mdlPart + 4, SourceModel.MaxBodyParts, "body group models"),
+ vtx.Count(vtxPart, SourceModel.MaxBodyParts, "body group models"));
+ for (var m = 0; m < models; m++)
+ {
+ var mdlModel = mdlPart + mdl.I32(mdlPart + 12) + m * 148;
+ var vtxModel = vtxPart + vtx.I32(vtxPart + 4) + m * 8;
+ var model = new BodyModel(mdl.String(mdlModel, 64), []);
+ var lods = vtx.Count(vtxModel, MaxLods, "LODs");
+ for (var l = 0; l < lods; l++)
+ {
+ var lod = vtxModel + vtx.I32(vtxModel + 4) + l * 12;
+ model.Lods.Add(new ModelLod(vtx.F32(lod + 8), SourceModel.ReadLodMeshes(mdl, vtx, mdlModel, lod,
+ Model.SkinMaterials, Model.Vertices.Count, Model.ExtendedVtx, cancellationToken)));
+ }
+ part.Models.Add(model);
+ }
+ BodyParts.Add(part);
+ }
+ }
+
+ private void ReadAttachments(SourceBytes mdl)
+ {
+ var count = mdl.Count(240, MaxAttachments, "attachments");
+ var start = mdl.I32(244);
+ for (var i = 0; i < count; i++)
+ {
+ var at = start + i * 92;
+ // A 3×4 matrix, row by row: the rotation's rows, each with the position's component last.
+ var rows = new float[12];
+ for (var k = 0; k < 12; k++)
+ rows[k] = mdl.F32(at + 12 + k * 4);
+ var rotation = new Matrix4x4(rows[0], rows[4], rows[8], 0, rows[1], rows[5], rows[9], 0, rows[2], rows[6], rows[10], 0, 0, 0, 0, 1);
+ Attachments.Add(new Attachment(mdl.RelativeString(at, at), mdl.I32(at + 8), new Vector3(rows[3], rows[7], rows[11]),
+ SourceAnimation.ToEuler(Quaternion.CreateFromRotationMatrix(rotation))));
+ }
+ }
+
+ private void ReadHitboxes(SourceBytes mdl)
+ {
+ var sets = mdl.Count(172, MaxAttachments, "hitbox sets");
+ var start = mdl.I32(176);
+ for (var s = 0; s < sets; s++)
+ {
+ var at = start + s * 12;
+ var set = new HitboxSet(mdl.RelativeString(at, at), []);
+ var boxes = mdl.Count(at + 4, MaxHitboxes, "hitboxes");
+ for (var h = 0; h < boxes; h++)
+ {
+ var box = at + mdl.I32(at + 8) + h * 68;
+ set.Boxes.Add(new Hitbox(mdl.I32(box), mdl.I32(box + 4), mdl.Vector(box + 8), mdl.Vector(box + 20)));
+ }
+ HitboxSets.Add(set);
+ }
+ }
+
+ private void ReadPoseParameters(SourceBytes mdl)
+ {
+ var count = mdl.Count(300, MaxAttachments, "pose parameters");
+ var start = mdl.I32(304);
+ for (var i = 0; i < count; i++)
+ {
+ var at = start + i * 20;
+ PoseParameters.Add(new PoseParameter(mdl.RelativeString(at, at), mdl.I32(at + 4), mdl.F32(at + 8), mdl.F32(at + 12), mdl.F32(at + 16)));
+ }
+ }
+
+ private void ReadIncludeModels(SourceBytes mdl)
+ {
+ var count = mdl.Count(336, MaxAttachments, "include models");
+ var start = mdl.I32(340);
+ for (var i = 0; i < count; i++)
+ IncludeModels.Add(mdl.RelativeString(start + i * 8, start + i * 8 + 4));
+ var blocks = mdl.Count(352, MaxAnimations, "animation blocks");
+ var table = mdl.I32(356);
+ for (var i = 0; i < blocks; i++)
+ AnimationBlocks.Add((mdl.I32(table + i * 8), mdl.I32(table + i * 8 + 4)));
+ }
+
+ private void ReadAnimations(SourceBytes mdl)
+ {
+ var count = mdl.Count(180, MaxAnimations, "animations");
+ var start = mdl.I32(184);
+ for (var i = 0; i < count; i++)
+ {
+ var at = start + i * 100;
+ var frames = mdl.I32(at + 16);
+ if (frames is < 0 or > SourceAnimation.MaxFrames)
+ throw new InvalidDataException($"{mdl.File} is damaged (an animation claims {frames} frames).");
+ Animations.Add(new Animation(at, mdl.RelativeString(at, at + 4), mdl.F32(at + 8), mdl.I32(at + 12), frames, mdl.I32(at + 84)));
+ }
+ }
+
+ private void ReadSequences(SourceBytes mdl, CancellationToken cancellationToken)
+ {
+ var count = mdl.Count(188, MaxAnimations, "sequences");
+ var start = mdl.I32(192);
+ for (var i = 0; i < count; i++)
+ {
+ cancellationToken.ThrowIfCancellationRequested();
+ var at = start + i * 212;
+ var events = new List();
+ var eventCount = mdl.Count(at + 24, MaxEvents, "animation events");
+ for (var e = 0; e < eventCount; e++)
+ {
+ var ev = at + mdl.I32(at + 28) + e * 80;
+ events.Add(new AnimationEvent(mdl.F32(ev), mdl.I32(ev + 4), mdl.I32(ev + 8), mdl.String(ev + 12, 64),
+ mdl.RelativeString(ev, ev + 76)));
+ }
+ var width = mdl.I32(at + 68);
+ var height = mdl.I32(at + 72);
+ if (width is < 1 or > MaxBlends || height is < 1 or > MaxBlends || (long)width * height > MaxBlends)
+ throw new InvalidDataException($"{mdl.File} is damaged (a sequence claims {width} × {height} blended animations).");
+ var table = at + mdl.I32(at + 60);
+ var animations = new List(width * height);
+ for (var k = 0; k < width * height; k++)
+ {
+ var index = mdl.I16(table + k * 2);
+ if (index < 0 || index >= Animations.Count)
+ throw new InvalidDataException($"{mdl.File} is damaged (a sequence uses animation {index}).");
+ animations.Add(index);
+ }
+ Sequences.Add(new Sequence(mdl.RelativeString(at, at + 4), mdl.RelativeString(at, at + 8), mdl.I32(at + 20),
+ mdl.I32(at + 12), events, width, animations, [mdl.I32(at + 76), mdl.I32(at + 80)],
+ [mdl.F32(at + 84), mdl.F32(at + 88)], [mdl.F32(at + 92), mdl.F32(at + 96)], mdl.F32(at + 104), mdl.F32(at + 108)));
+ }
+ }
+}
diff --git a/src/Filee.Engines/Models3D/SourcePhysics.cs b/src/Filee.Engines/Models3D/SourcePhysics.cs
new file mode 100644
index 000000000..a64156016
--- /dev/null
+++ b/src/Filee.Engines/Models3D/SourcePhysics.cs
@@ -0,0 +1,151 @@
+// A Source engine model's physics (.phy, #1526): the convex pieces of its collision model and the settings studiomdl
+// took from the QC, read to decompile them into a physics SMD and the QC's $collisionmodel or $collisionjoints.
+// Layout (little endian), learned from Valve's vcollide and IVP headers and SourceIO's reader:
+//
+// header 16 bytes: its size, id, the number of solids, the checksum of the .mdl
+// solid its size, then "VPHY" with version, type, surface size, drag areas, axis map size (28 bytes) —
+// older files start straight with the surface — and an IVP compact surface:
+// compact surface 48 bytes: mass centre, inertia, radius, size, the offset of its ledge tree, "IVPS" at 44
+// ledge tree nodes of 28 bytes: offset of the right child (0 for a leaf), offset of the leaf's ledge, centre,
+// radius, box; the left child follows the node
+// ledge a convex piece, 16 bytes: offset of its points, the bone (+1) it belongs to, flags, triangles;
+// then 16 bytes a triangle: index, three edges of (point, opposite) shorts
+// points 16 bytes: x, y, z in metres in IVP's axes (Source x = x, y = z, z = −y), and one unused float
+// key values after the solids: text blocks such as solid { "name" "mass" … }, ragdollconstraint { … },
+// editparams { "rootname" "totalmass" "concave" }
+//
+// The pieces of a solid named after a bone ($collisionjoints, a ragdoll) are in that bone's space; others are in the
+// model's. Every count is capped and every offset checked against the file; the ledge tree is walked without
+// recursion and stops at a node count, so a looping tree is a damaged file.
+
+using System.Numerics;
+
+namespace Filee.Engines.Models3D;
+
+/// A Source model's .phy: collision pieces and their settings.
+internal sealed class SourcePhysics
+{
+ private const int MaxSolids = 1024;
+ private const int MaxNodes = 65536;
+ private const int MaxPoints = 65536;
+
+ /// Metres (IVP's unit) to inches (Source's).
+ private const float InchesPerMetre = 1 / 0.0254f;
+
+ /// A convex piece: its points (Source's axes, inches) and triangles (indices into them, counter-clockwise outward).
+ internal sealed record Piece(List Points, List Triangles);
+
+ /// A solid: its pieces and the bone number the file stores with them (−1 for none).
+ internal sealed record Solid(List Pieces, int Bone);
+
+ /// A key-value block of the text section, such as "solid" or "ragdollconstraint".
+ internal sealed record Block(string Name, Dictionary Values);
+
+ public int Checksum { get; private init; }
+ public List Solids { get; } = [];
+ public List Blocks { get; } = [];
+
+ /// Reads the .phy at .
+ public static SourcePhysics Read(string path)
+ {
+ var phy = SourceBytes.Read(path);
+ var headerSize = phy.I32(0);
+ var solids = phy.Count(8, MaxSolids, "solids");
+ var physics = new SourcePhysics { Checksum = phy.I32(12) };
+ var at = headerSize;
+ for (var s = 0; s < solids; s++)
+ {
+ var size = phy.I32(at);
+ if (size <= 0 || !phy.Fits(at + 4, size, 1))
+ throw phy.Damaged("a solid");
+ physics.Solids.Add(ReadSolid(phy, at + 4));
+ at += 4 + size;
+ }
+ if (at < phy.Data.Length)
+ physics.Blocks.AddRange(ReadBlocks(phy.String(at, phy.Data.Length - at)));
+ return physics;
+ }
+
+ private static Solid ReadSolid(SourceBytes phy, int at)
+ {
+ var surface = phy.Data.AsSpan(at).StartsWith("VPHY"u8) ? at + 28 : at;
+ if (!phy.Fits(surface, 48, 1) || !phy.Data.AsSpan(surface + 44, 4).SequenceEqual("IVPS"u8))
+ throw new InvalidDataException($"{phy.File} has a solid Filee doesn't read (no IVP compact surface).");
+ var root = surface + phy.I32(surface + 32);
+ var pieces = new List();
+ var bone = -1;
+ var stack = new Stack();
+ stack.Push(root);
+ for (var visited = 0; stack.Count > 0; visited++)
+ {
+ if (visited > MaxNodes)
+ throw new InvalidDataException($"{phy.File} is damaged (its collision tree doesn't end).");
+ var node = stack.Pop();
+ var right = phy.I32(node);
+ if (right != 0)
+ {
+ stack.Push(node + right);
+ stack.Push(node + 28);
+ continue;
+ }
+ var ledge = node + phy.I32(node + 4);
+ pieces.Add(ReadPiece(phy, ledge));
+ bone = phy.I32(ledge + 4) - 1;
+ }
+ return new Solid(pieces, bone);
+ }
+
+ /// A convex piece (IVP compact ledge) at , in Source's axes and inches.
+ private static Piece ReadPiece(SourceBytes phy, int ledge)
+ {
+ var pointTable = ledge + phy.I32(ledge);
+ var triangles = phy.I16(ledge + 12);
+ if (triangles < 0 || !phy.Fits(ledge + 16, triangles, 16))
+ throw phy.Damaged("a collision piece");
+ var points = new List();
+ var local = new Dictionary();
+ var indices = new List(triangles * 3);
+ for (var t = 0; t < triangles; t++)
+ {
+ for (var e = 0; e < 3; e++)
+ {
+ var point = phy.U16(ledge + 16 + t * 16 + 4 + e * 4);
+ if (!local.TryGetValue(point, out var index))
+ {
+ if (points.Count >= MaxPoints)
+ throw new InvalidDataException($"{phy.File} is damaged (a collision piece claims too many points).");
+ var p = pointTable + point * 16;
+ var ivp = phy.Vector(p);
+ index = points.Count;
+ local[point] = index;
+ points.Add(new Vector3(ivp.X, ivp.Z, -ivp.Y) * InchesPerMetre);
+ }
+ indices.Add(index);
+ }
+ }
+ return new Piece(points, indices);
+ }
+
+ /// The text section's blocks: a name, then "{", then quoted key and value pairs, then "}".
+ internal static List ReadBlocks(string text)
+ {
+ var tokens = Vmt.Tokens(text);
+ var blocks = new List();
+ for (var i = 0; i + 1 < tokens.Count; i++)
+ {
+ if (tokens[i + 1] != "{")
+ continue;
+ var values = new Dictionary(StringComparer.OrdinalIgnoreCase);
+ var j = i + 2;
+ for (; j + 1 < tokens.Count && tokens[j] != "}"; j += 2)
+ {
+ if (tokens[j + 1] is "{" or "}")
+ break;
+ values[tokens[j]] = tokens[j + 1];
+ }
+ blocks.Add(new Block(tokens[i], values));
+ i = j;
+ }
+ return blocks;
+ }
+}
diff --git a/tests/Filee.Core.Tests/FormatFieldsTests.cs b/tests/Filee.Core.Tests/FormatFieldsTests.cs
index 0908ad978..bc11aac35 100644
--- a/tests/Filee.Core.Tests/FormatFieldsTests.cs
+++ b/tests/Filee.Core.Tests/FormatFieldsTests.cs
@@ -26,16 +26,16 @@ private static string[] Hidden(IReadOnlySet turnedOff) =>
private static readonly string[] HiddenWithEverythingOff =
[
"3g2", "3gp", "3mf", "602", "aab", "ac3", "adoc", "ai", "amr", "ani", "ape", "apks", "arj", "ass", "au", "azw",
- "azw4", "bik", "bz2", "cab", "caf", "cavs", "cdr", "cgm", "chm", "cpio", "cur", "cwk", "dav", "dbk", "dcm", "dds", "deb",
+ "azw4", "bik", "bsp", "bz2", "cab", "caf", "cavs", "cdr", "cgm", "chm", "cpio", "cur", "cwk", "dav", "dbk", "dcm", "dds", "deb",
"dff", "dot", "dsf", "dss", "dst", "dts", "dv", "dwg", "eac3", "eot", "eps", "exr", "fbx", "fit", "fits",
- "flc", "flv", "fodg", "fodp", "fods", "fodt", "fon", "gcode", "geojson", "gerber", "glb", "gltf", "gpx", "gz", "h264", "h265", "har",
+ "flc", "flv", "fodg", "fodp", "fods", "fodt", "fon", "gcode", "geojson", "gerber", "glb", "gltf", "gma", "gpx", "gz", "h264", "h265", "har",
"hdr", "htmlz", "icns", "ipynb", "it", "jar", "jp2", "jsonl", "kml", "kmz", "kra", "lha", "lit", "lrf", "lwp", "lz",
- "lzma", "m4s", "man", "mcaddon", "mcpack", "mctemplate", "mcw", "mcworld", "mediawiki", "miff", "mng", "mobi",
+ "lzma", "m4s", "man", "mcaddon", "mcpack", "mctemplate", "mcw", "mcworld", "mdl", "mediawiki", "miff", "mng", "mobi",
"mod", "mp2", "mpc", "mrpack", "mxf", "nov", "nupkg", "obj", "oeb",
"org", "osm", "otf", "pam", "pcd", "pcx", "pdb", "pdn", "pfm", "plist", "plt", "ply", "pml", "ppm", "prc", "ps", "psb", "psd",
- "psw", "qoi", "ras", "raw", "rb", "rm", "rpm", "rst", "s3m", "scc", "sda", "sdc", "sdw", "sgi", "shp", "snb", "spx",
+ "psw", "qoi", "ras", "raw", "rb", "rm", "rpm", "rst", "s3m", "scc", "sda", "sdc", "sdw", "sgi", "shp", "smd", "snb", "spx",
"ssa", "stl", "sub", "swf", "tak", "tar", "tar7z", "tbz2", "tcr", "tex", "textile", "tgz", "tlz", "toml",
- "tta", "ttf", "ttml", "txtz", "txz", "typ", "tz", "usm", "vob", "voc", "vtf", "w64", "wbmp", "wmf", "woff", "woff2",
+ "tta", "ttf", "ttml", "txtz", "txz", "typ", "tz", "usm", "vob", "voc", "vpk", "vtf", "w64", "wbmp", "wmf", "woff", "woff2",
"wpd", "wtv", "wv", "xbm", "xcf", "xlt", "xm", "xpm", "xz", "yaml", "z", "zxp"
];
@@ -45,7 +45,7 @@ private static string[] Hidden(IReadOnlySet turnedOff) =>
{ "model3d", ["fbx", "glb", "gltf", "ply"] },
{ "geo", ["fit", "geojson", "gpx", "kml", "kmz", "osm", "shp"] },
{ "font", ["otf", "ttf", "woff", "woff2"] },
- { "game", ["bik", "dds", "mcaddon", "mcpack", "mctemplate", "mcworld", "mrpack", "usm", "vtf"] },
+ { "game", ["bik", "bsp", "dds", "gma", "mcaddon", "mcpack", "mctemplate", "mcworld", "mdl", "mrpack", "smd", "usm", "vpk", "vtf"] },
{ "design", ["ai", "cdr", "dst", "eps", "exr", "hdr", "kra", "pdn", "plt", "ps", "psb", "psd", "raw", "xcf"] },
{ "science", ["dcm", "fits"] },
{ "developer", [
@@ -72,7 +72,7 @@ private static string[] Hidden(IReadOnlySet turnedOff) =>
{ "model3d", ["3mf", "fbx", "gcode", "glb", "gltf", "obj", "ply", "stl"] },
{ "geo", ["fit", "geojson", "gpx", "kml", "kmz", "osm", "shp"] },
{ "font", ["eot", "fon", "otf", "ttf", "woff", "woff2"] },
- { "game", ["bik", "dds", "flc", "it", "mcaddon", "mcpack", "mctemplate", "mcworld", "mrpack", "tga", "usm", "vtf", "xm"] },
+ { "game", ["bik", "bsp", "dds", "flc", "gma", "it", "mcaddon", "mcpack", "mctemplate", "mcworld", "mdl", "mrpack", "smd", "tga", "usm", "vpk", "vtf", "xm"] },
{ "design", ["ai", "cdr", "dst", "eps", "exr", "hdr", "jp2", "kra", "pcd", "pdn", "plt", "ps", "psb", "psd", "raw", "xcf"] },
{ "science", ["dcm", "fits", "ipynb", "jp2"] },
{ "developer", [
@@ -145,7 +145,7 @@ public void Grade_counts_match_the_published_table()
{
[FormatGrade.Everyday] = 19,
[FormatGrade.General] = 99,
- [FormatGrade.Specialist] = 127,
+ [FormatGrade.Specialist] = 132,
[FormatGrade.Legacy] = 52,
},
counts);
diff --git a/tests/Filee.Core.Tests/SourceFormatTests.cs b/tests/Filee.Core.Tests/SourceFormatTests.cs
new file mode 100644
index 000000000..5886afde8
--- /dev/null
+++ b/tests/Filee.Core.Tests/SourceFormatTests.cs
@@ -0,0 +1,126 @@
+// Source engine files (#1526): Garry's Mod addons, Valve paks and maps (for their packed files) open the archives donut,
+// which existing libraries learn without taking a donut of the user's own, and Source models and SMD the 3D models
+// donut; each is a target only where the Games field is on.
+
+using Filee.Core.Formats;
+using Filee.Core.Presets;
+using Filee.Core.Profiles;
+using Filee.Core.Settings;
+
+namespace Filee.Core.Tests;
+
+public class SourceFormatTests
+{
+ [Theory]
+ [InlineData("wire.gma", "gma", "archives")]
+ [InlineData("WIRE.GMA", "gma", "archives")]
+ [InlineData("pak01_dir.vpk", "vpk", "archives")]
+ [InlineData("gm_construct.bsp", "bsp", "archives")]
+ [InlineData("gate_e2.mdl", "mdl", "models")]
+ [InlineData("gate_e2_reference.smd", "smd", "models")]
+ public void Source_files_use_their_donut(string file, string format, string donut)
+ {
+ Assert.Same(FormatRegistry.Get(format), FormatRegistry.Detect(file));
+ Assert.Equal(donut, ProfileSelector.Select(BuiltInData.CreateProfiles(), [file])?.Id);
+ }
+
+ [Theory]
+ [InlineData("gma", FormatCategory.Archive, true)]
+ [InlineData("vpk", FormatCategory.Archive, false)]
+ [InlineData("bsp", FormatCategory.Archive, false)]
+ [InlineData("mdl", FormatCategory.Model3D, false)]
+ [InlineData("smd", FormatCategory.Model3D, true)]
+ public void Source_files_are_game_formats(string id, FormatCategory category, bool writable)
+ {
+ var format = FormatRegistry.Get(id)!;
+ Assert.Equal((category, writable), (format.Category, format.Writable));
+ Assert.Equal(FormatGrade.Specialist, FormatFields.GradeOf(format));
+ Assert.False(FormatFields.IsShown(id, new HashSet { "game" }));
+ Assert.True(FormatFields.IsShown(id, new HashSet()));
+ }
+
+ [Fact]
+ public void Gma_joins_the_game_packs_and_smd_the_3d_models_in_the_palette()
+ {
+ var presets = BuiltInData.CreatePresets();
+ var ids = presets.Select(p => p.Id).ToList();
+ Assert.Equal(ids.IndexOf("to-mrpack") + 1, ids.IndexOf("to-gma"));
+ Assert.Equal(ids.IndexOf("to-gltf") + 1, ids.IndexOf("to-smd"));
+ Assert.Equal("gma", presets.Single(p => p.Id == "to-gma").TargetFormat);
+ Assert.Equal("smd", presets.Single(p => p.Id == "to-smd").TargetFormat);
+ }
+
+ [Fact]
+ public void Version_94_libraries_learn_the_source_files_without_taking_custom_claims()
+ {
+ var presets = BuiltInData.CreatePresets();
+ var profiles = BuiltInData.CreateProfiles();
+ var archives = profiles.Single(p => p.Id == "archives");
+ archives.Extensions.RemoveAll(e => e is "gma" or "vpk" or "bsp");
+
+ Assert.True(SettingsMigrations.ApplyToLibrary(94, presets, profiles));
+ Assert.Contains("gma", archives.Extensions);
+ Assert.Contains("vpk", archives.Extensions);
+ Assert.Contains("bsp", archives.Extensions);
+ Assert.Equal(BuiltInData.CreatePresets().Select(p => p.Id), presets.Select(p => p.Id)); // no preset added
+ Assert.False(SettingsMigrations.ApplyToLibrary(94, presets, profiles));
+
+ archives.Extensions.Remove("gma");
+ profiles.Insert(0, new ToolbarProfile { Id = "my-addons", Extensions = ["GMA"] });
+ Assert.False(SettingsMigrations.ApplyToLibrary(94, presets, profiles));
+ Assert.DoesNotContain("gma", archives.Extensions); // the user's own donut has it
+ Assert.Equal("my-addons", ProfileSelector.Select(profiles, ["wire.gma"])?.Id);
+ }
+
+ [Fact]
+ public void Version_95_libraries_get_the_gma_preset_after_the_game_packs()
+ {
+ var presets = BuiltInData.CreatePresets();
+ presets.RemoveAll(p => p.Id == "to-gma");
+ var profiles = BuiltInData.CreateProfiles();
+
+ Assert.True(SettingsMigrations.ApplyToLibrary(95, presets, profiles));
+ var ids = presets.Select(p => p.Id).ToList();
+ Assert.Equal(ids.IndexOf("to-mrpack") + 1, ids.IndexOf("to-gma"));
+ Assert.False(SettingsMigrations.ApplyToLibrary(95, presets, profiles));
+ // Not on a donut.
+ Assert.DoesNotContain(BuiltInData.CreateProfiles(), p => p.PresetIds.Contains("to-gma") || p.BackPresetIds?.Contains("to-gma") == true);
+ }
+
+ [Fact]
+ public void Version_96_libraries_learn_source_models_and_get_the_smd_preset()
+ {
+ var presets = BuiltInData.CreatePresets();
+ var profiles = BuiltInData.CreateProfiles();
+ var models = profiles.Single(p => p.Id == "models");
+ models.Extensions.RemoveAll(e => e is "mdl" or "smd");
+ presets.RemoveAll(p => p.Id == "to-smd");
+
+ Assert.True(SettingsMigrations.ApplyToLibrary(96, presets, profiles));
+ Assert.Contains("mdl", models.Extensions);
+ Assert.Contains("smd", models.Extensions);
+ Assert.Equal(BuiltInData.CreatePresets().Select(p => p.Id), presets.Select(p => p.Id)); // "SMD" back after glTF
+ Assert.False(SettingsMigrations.ApplyToLibrary(96, presets, profiles));
+
+ models.Extensions.Remove("mdl");
+ profiles.Insert(0, new ToolbarProfile { Id = "my-models", Extensions = ["MDL"] });
+ Assert.False(SettingsMigrations.ApplyToLibrary(96, presets, profiles));
+ Assert.DoesNotContain("mdl", models.Extensions);
+ }
+
+ [Fact]
+ public void Version_97_libraries_get_the_decompile_preset_after_smd()
+ {
+ var presets = BuiltInData.CreatePresets();
+ presets.RemoveAll(p => p.Id == BuiltInData.DecompileModelId);
+ var profiles = BuiltInData.CreateProfiles();
+
+ Assert.True(SettingsMigrations.ApplyToLibrary(97, presets, profiles));
+ Assert.Equal(BuiltInData.CreatePresets().Select(p => p.Id), presets.Select(p => p.Id)); // right after "SMD"
+ Assert.Equal(FormatRegistry.Folder, presets.Single(p => p.Id == BuiltInData.DecompileModelId).TargetFormat);
+ Assert.False(SettingsMigrations.ApplyToLibrary(97, presets, profiles));
+ // Not on a donut.
+ Assert.DoesNotContain(BuiltInData.CreateProfiles(),
+ p => p.PresetIds.Contains(BuiltInData.DecompileModelId) || p.BackPresetIds?.Contains(BuiltInData.DecompileModelId) == true);
+ }
+}
diff --git a/tests/Filee.Core.Tests/TranslatePresetMigrationTests.cs b/tests/Filee.Core.Tests/TranslatePresetMigrationTests.cs
index 5ae8ea156..6f07db16e 100644
--- a/tests/Filee.Core.Tests/TranslatePresetMigrationTests.cs
+++ b/tests/Filee.Core.Tests/TranslatePresetMigrationTests.cs
@@ -127,7 +127,7 @@ public void Translate_goes_after_the_built_in_preset_before_it()
var defaults = BuiltInData.CreatePresets();
List Only(params string[] ids) => ids.Select(id => defaults.Single(p => p.Id == id)).ToList();
// Later steps add their own presets; what is checked here is where Translate lands.
- static IEnumerable Ids(List presets) => presets.Select(p => p.Id).Where(id => id is not (BuiltInData.MetadataId or BuiltInData.UpscaleId or BuiltInData.RingtoneId or BuiltInData.PdfAId or "to-iso"));
+ static IEnumerable Ids(List presets) => presets.Select(p => p.Id).Where(id => id is not (BuiltInData.MetadataId or BuiltInData.UpscaleId or BuiltInData.RingtoneId or BuiltInData.PdfAId or "to-iso" or "to-gma" or "to-smd" or BuiltInData.DecompileModelId));
List Profiles()
{
var profiles = BuiltInData.CreateProfiles();
diff --git a/tests/Filee.Core.Tests/UserDataStoreTests.cs b/tests/Filee.Core.Tests/UserDataStoreTests.cs
index 887570306..39a51415a 100644
--- a/tests/Filee.Core.Tests/UserDataStoreTests.cs
+++ b/tests/Filee.Core.Tests/UserDataStoreTests.cs
@@ -1060,7 +1060,7 @@ public void Version_14_libraries_get_3d_model_presets_and_their_donut_after_data
Assert.Equal(ids.IndexOf("models") + 1, ids.IndexOf("geo")); // and, from schema 25, the geospatial donut
Assert.Equal(ids.IndexOf("mixed") - 1, ids.IndexOf("geo"));
var models = profiles.Single(p => p.Id == "models");
- Assert.Equal(["glb", "gltf", "stl", "obj", "3mf", "ply", "fbx"], models.Extensions);
+ Assert.Equal(["glb", "gltf", "stl", "obj", "3mf", "ply", "fbx", "mdl", "smd"], models.Extensions);
Assert.Equal(BuiltInData.ModelProfile().PresetIds, models.PresetIds);
Assert.False(SettingsMigrations.ApplyToLibrary(14, presets, profiles)); // once only
}
@@ -1074,7 +1074,7 @@ public void The_3d_donut_leaves_out_extensions_a_profile_of_the_user_takes()
Assert.True(SettingsMigrations.ApplyToLibrary(14, presets, profiles));
- Assert.Equal(["glb", "gltf", "obj", "ply", "fbx"], profiles.Single(p => p.Id == "models").Extensions);
+ Assert.Equal(["glb", "gltf", "obj", "ply", "fbx", "mdl", "smd"], profiles.Single(p => p.Id == "models").Extensions);
}
[Fact]
@@ -1088,7 +1088,7 @@ public void Version_21_libraries_learn_fbx_on_the_3d_donut_and_nothing_else()
Assert.True(SettingsMigrations.ApplyToLibrary(21, presets, profiles));
- Assert.Equal(["glb", "gltf", "stl", "obj", "3mf", "fbx"], models.Extensions);
+ Assert.Equal(["glb", "gltf", "stl", "obj", "3mf", "mdl", "smd", "fbx"], models.Extensions);
Assert.False(SettingsMigrations.ApplyToLibrary(21, presets, profiles)); // once only
// A donut of the user that already opens FBX files keeps them.
diff --git a/tests/Filee.Engines.Tests/BspTests.cs b/tests/Filee.Engines.Tests/BspTests.cs
new file mode 100644
index 000000000..dcc69b5ef
--- /dev/null
+++ b/tests/Filee.Engines.Tests/BspTests.cs
@@ -0,0 +1,143 @@
+// Source engine maps (.bsp, #1526): the files packed into a map are a ZIP in its pakfile lump (lump 40). It unpacks like
+// an archive; Left 4 Dead 2's lump order (version, offset, length) is read too. The test maps are built here: a header,
+// some lump data and the ZIP.
+
+using System.Buffers.Binary;
+using System.IO.Compression;
+using Filee.Core.Conversion;
+using Filee.Core.Formats;
+using Filee.Core.Presets;
+using Filee.Engines.Archives;
+using static Filee.Engines.Tests.ArchiveBuilders;
+
+namespace Filee.Engines.Tests;
+
+public class BspTests(EngineFixture fx) : IClassFixture
+{
+ private static CancellationToken Ct => TestContext.Current.CancellationToken;
+
+ private void SkipWithout7Zip()
+ {
+ var engine = fx.Converters.Single(c => c.Id == "archive");
+ Assert.SkipUnless(fx.Catalog.StatusOf(engine).IsAvailable, "7-Zip not found (pwsh build/fetch-engines.ps1 -Only 7zip)");
+ }
+
+ private async Task ConvertOkAsync(string source, string presetId)
+ {
+ var job = await fx.ConvertAsync([source], BuiltInData.CreatePresets().Single(p => p.Id == presetId));
+ Assert.True(job.State == JobState.Completed, string.Join("; ", job.Files.Select(f => $"{f.ErrorKey} {f.ErrorDetail}")));
+ return Assert.Single(job.Outputs);
+ }
+
+ /// A ZIP as VBSP packs it: stored, with the map's custom files.
+ private static byte[] PakfileZip(params (string Name, byte[] Data)[] files)
+ {
+ using var stream = new MemoryStream();
+ using (var zip = new ZipArchive(stream, ZipArchiveMode.Create, leaveOpen: true))
+ {
+ foreach (var (name, data) in files)
+ {
+ using var entry = zip.CreateEntry(name, CompressionLevel.NoCompression).Open();
+ entry.Write(data);
+ }
+ }
+ return stream.ToArray();
+ }
+
+ ///
+ /// A map of version : the header, 4 KB of other lump data, then
+ /// as lump 40 (in Left 4 Dead 2's order when ).
+ ///
+ private static byte[] BuildMap(int version, byte[] pakfile, bool l4d2 = false)
+ {
+ const int headerSize = 8 + 64 * 16 + 4;
+ var other = Noise(4096, 7);
+ var map = new byte[headerSize + other.Length + pakfile.Length];
+ "VBSP"u8.CopyTo(map);
+ BinaryPrimitives.WriteInt32LittleEndian(map.AsSpan(4), version);
+ // Lump 0 (entities) points at the other data, as in a real map.
+ BinaryPrimitives.WriteInt32LittleEndian(map.AsSpan(8), headerSize);
+ BinaryPrimitives.WriteInt32LittleEndian(map.AsSpan(12), other.Length);
+ var lump = map.AsSpan(8 + 40 * 16, 16);
+ var offset = headerSize + other.Length;
+ if (l4d2)
+ {
+ BinaryPrimitives.WriteInt32LittleEndian(lump, 0);
+ BinaryPrimitives.WriteInt32LittleEndian(lump[4..], offset);
+ BinaryPrimitives.WriteInt32LittleEndian(lump[8..], pakfile.Length);
+ }
+ else
+ {
+ BinaryPrimitives.WriteInt32LittleEndian(lump, offset);
+ BinaryPrimitives.WriteInt32LittleEndian(lump[4..], pakfile.Length);
+ }
+ other.CopyTo(map, headerSize);
+ pakfile.CopyTo(map, offset);
+ return map;
+ }
+
+ private static readonly (string Name, byte[] Data)[] Packed =
+ [
+ ("materials/maps/gm_test/brick.vtf", Noise(2000, 1)),
+ ("materials/maps/gm_test/brick.vmt", "\"LightmappedGeneric\" { \"$basetexture\" \"maps/gm_test/brick\" }"u8.ToArray()),
+ ("sound/ambient/wind.wav", Noise(800, 2)),
+ ];
+
+ [Fact]
+ public void Bsp_is_recognised_and_unpacks_in_one_step()
+ {
+ Assert.Equal("bsp", FormatRegistry.Detect("gm_construct.BSP")?.Id);
+ var planner = fx.Catalog.CreatePlanner(["archive"]);
+ Assert.Single(planner.Plan("bsp", FormatRegistry.Folder)!.Steps);
+ Assert.Single(planner.Plan("bsp", "zip")!.Steps);
+ }
+
+ [Theory]
+ [InlineData(20, false)]
+ [InlineData(21, true)]
+ public async Task A_maps_packed_files_unpack(int version, bool l4d2)
+ {
+ SkipWithout7Zip();
+ var path = Path.Combine(fx.NewFolder(), "gm_test.bsp");
+ File.WriteAllBytes(path, BuildMap(version, PakfileZip(Packed), l4d2));
+
+ var folder = await ConvertOkAsync(path, "extract");
+
+ foreach (var (name, data) in Packed)
+ Assert.Equal(data, File.ReadAllBytes(Path.Combine(folder, name)));
+ }
+
+ [Fact]
+ public async Task A_maps_packed_files_become_a_zip()
+ {
+ SkipWithout7Zip();
+ var path = Path.Combine(fx.NewFolder(), "gm_test.bsp");
+ File.WriteAllBytes(path, BuildMap(20, PakfileZip(Packed)));
+
+ var zip = await ConvertOkAsync(path, "to-zip");
+
+ using var archive = ZipFile.OpenRead(zip);
+ Assert.Equal(Packed.Select(p => p.Name).Order(StringComparer.Ordinal),
+ archive.Entries.Where(e => !e.FullName.EndsWith('/')).Select(e => e.FullName).Order(StringComparer.Ordinal));
+ }
+
+ [Fact]
+ public void A_map_without_packed_files_and_a_quake_map_are_refused_clearly()
+ {
+ var folder = fx.NewFolder();
+
+ var empty = Path.Combine(folder, "empty.bsp");
+ File.WriteAllBytes(empty, BuildMap(20, PakfileZip()));
+ var error = Assert.Throws(() => Bsp.ExtractPakfile(empty, Path.Combine(folder, "a.zip"), Ct));
+ Assert.Contains("no files packed", error.Message);
+
+ var quake = Path.Combine(folder, "e1m1.bsp");
+ File.WriteAllBytes(quake, [29, 0, 0, 0, .. Noise(2000, 3)]); // Quake's BSP starts with its version
+ error = Assert.Throws(() => Bsp.ExtractPakfile(quake, Path.Combine(folder, "b.zip"), Ct));
+ Assert.Contains("not a Source engine map", error.Message);
+
+ var cut = Path.Combine(folder, "cut.bsp");
+ File.WriteAllBytes(cut, BuildMap(20, PakfileZip(Packed))[..5000]);
+ Assert.Throws(() => Bsp.ExtractPakfile(cut, Path.Combine(folder, "c.zip"), Ct));
+ }
+}
diff --git a/tests/Filee.Engines.Tests/GmaPackTests.cs b/tests/Filee.Engines.Tests/GmaPackTests.cs
new file mode 100644
index 000000000..c43bef291
--- /dev/null
+++ b/tests/Filee.Engines.Tests/GmaPackTests.cs
@@ -0,0 +1,199 @@
+// Packing Garry's Mod addons (.gma, #1526): an addon folder packs into a GMA as gmad does (addon.json for the title, type,
+// tags and ignores; only the files Garry's Mod loads; lower-case names; a CRC per file and over the whole addon), and an
+// unpacked addon packs back into the same addon.
+
+using System.Buffers.Binary;
+using System.IO.Compression;
+using System.Text;
+using System.Text.Json.Nodes;
+using Filee.Core.Conversion;
+using Filee.Core.Presets;
+using Filee.Engines.Archives;
+using static Filee.Engines.Tests.ArchiveBuilders;
+
+namespace Filee.Engines.Tests;
+
+public class GmaPackTests(EngineFixture fx) : IClassFixture
+{
+ private async Task ConvertOkAsync(IReadOnlyList sources, Preset preset)
+ {
+ var job = await fx.ConvertAsync(sources, preset);
+ Assert.True(job.State == JobState.Completed, string.Join("; ", job.Files.Select(f => $"{f.ErrorKey} {f.ErrorDetail}")));
+ return Assert.Single(job.Outputs);
+ }
+
+ private static Preset Extract => BuiltInData.CreatePresets().Single(p => p.Id == "extract");
+
+ /// An addon folder: addon.json, a Lua script, a texture, a model part, and files Garry's Mod doesn't load.
+ private static string AddonFolder(string parent, string name, bool withJson = true)
+ {
+ var folder = Directory.CreateDirectory(Path.Combine(parent, name)).FullName;
+ if (withJson)
+ {
+ File.WriteAllText(Path.Combine(folder, "addon.json"), """
+ {
+ "title": "Wire Gates",
+ "type": "Tool",
+ "tags": ["Fun", "build"],
+ "description": "Logic gates.",
+ "ignore": ["*.psd", "src/*"]
+ }
+ """);
+ }
+ Write(folder, "lua/autorun/Gates.lua", Encoding.UTF8.GetBytes("print('gates')\n"));
+ Write(folder, "materials/gates/gate.vtf", Noise(300, 1));
+ Write(folder, "materials/gates/gate.vmt", Encoding.UTF8.GetBytes("\"VertexLitGeneric\" { \"$basetexture\" \"gates/gate\" }"));
+ Write(folder, "models/gates/gate.mdl", Noise(200, 2));
+ Write(folder, "models/gates/gate.sw.vtx", Noise(50, 3)); // unused by the game: whitelist takes it back
+ Write(folder, "materials/gates/gate.psd", Noise(100, 4)); // ignored by addon.json
+ Write(folder, "readme.txt", Encoding.UTF8.GetBytes("not loaded by the game"));
+ return folder;
+ }
+
+ private static void Write(string folder, string name, byte[] data)
+ {
+ var path = Path.Combine(folder, name);
+ Directory.CreateDirectory(Path.GetDirectoryName(path)!);
+ File.WriteAllBytes(path, data);
+ }
+
+ private static Gma.Header ReadHeader(string gma)
+ {
+ using var stream = File.OpenRead(gma);
+ return Gma.ReadHeader(stream, Path.GetFileName(gma));
+ }
+
+ [Fact]
+ public async Task An_addon_folder_packs_as_gmad_does()
+ {
+ var folder = AddonFolder(fx.NewFolder(), "wire-gates");
+
+ var gma = await ConvertOkAsync([folder], new Preset { TargetFormat = "gma", Archive = { CombineIntoOne = true } });
+
+ Assert.Equal("wire-gates.gma", Path.GetFileName(gma));
+ var bytes = File.ReadAllBytes(gma);
+ Assert.Equal("GMAD"u8.ToArray(), bytes[..4]);
+ Assert.Equal(3, bytes[4]);
+ // The CRC of everything before the last four bytes.
+ Assert.Equal(Crc32.Compute(bytes.AsSpan(0, bytes.Length - 4)), BinaryPrimitives.ReadUInt32LittleEndian(bytes.AsSpan(bytes.Length - 4)));
+
+ var header = ReadHeader(gma);
+ Assert.Equal("Wire Gates", header.Title);
+ Assert.Equal("Logic gates.", header.Description);
+ Assert.Equal("tool", header.Type);
+ Assert.Equal(new[] { "fun", "build" }, header.Tags);
+ // Lower case, sorted, only what Garry's Mod loads: no addon.json, no .sw.vtx, no ignored .psd, no readme.
+ Assert.Equal(
+ ["lua/autorun/gates.lua", "materials/gates/gate.vmt", "materials/gates/gate.vtf", "models/gates/gate.mdl"],
+ header.Entries.Select(e => e.Name));
+ var texture = header.Entries.Single(e => e.Name.EndsWith(".vtf", StringComparison.Ordinal));
+ Assert.Equal(300, texture.Size);
+ Assert.Equal(Crc32.Compute(Noise(300, 1)), texture.Crc);
+ }
+
+ [Fact]
+ public async Task A_folder_without_addon_json_takes_its_name_as_the_title()
+ {
+ var folder = AddonFolder(fx.NewFolder(), "My Props", withJson: false);
+
+ var header = ReadHeader(await ConvertOkAsync([folder], new Preset { TargetFormat = "gma", Archive = { CombineIntoOne = true } }));
+
+ Assert.Equal("My Props", header.Title);
+ Assert.Equal("servercontent", header.Type);
+ // Without addon.json nothing is ignored by it, but the whitelist still applies.
+ Assert.DoesNotContain(header.Entries, e => e.Name.EndsWith(".psd", StringComparison.Ordinal));
+ Assert.Contains(header.Entries, e => e.Name == "models/gates/gate.mdl");
+ }
+
+ [Fact]
+ public async Task A_folder_with_nothing_garrys_mod_loads_is_refused()
+ {
+ var folder = Directory.CreateDirectory(Path.Combine(fx.NewFolder(), "notes")).FullName;
+ File.WriteAllText(Path.Combine(folder, "readme.txt"), "hello");
+
+ var job = await fx.ConvertAsync([folder], new Preset { TargetFormat = "gma", Archive = { CombineIntoOne = true } });
+
+ Assert.Equal(JobState.Failed, job.State);
+ Assert.Empty(job.Outputs);
+ }
+
+ [Fact]
+ public async Task An_addon_unpacks_and_packs_back_into_the_same_addon()
+ {
+ SkipWithout7Zip();
+ var folder = AddonFolder(fx.NewFolder(), "wire-gates");
+ var gma = await ConvertOkAsync([folder], new Preset { TargetFormat = "gma", Archive = { CombineIntoOne = true } });
+
+ var unpacked = await ConvertOkAsync([gma], Extract);
+
+ Assert.Equal(Encoding.UTF8.GetBytes("print('gates')\n"), File.ReadAllBytes(Path.Combine(unpacked, "lua", "autorun", "gates.lua")));
+ Assert.Equal(Noise(300, 1), File.ReadAllBytes(Path.Combine(unpacked, "materials", "gates", "gate.vtf")));
+ var json = JsonNode.Parse(File.ReadAllText(Path.Combine(unpacked, "addon.json")))!;
+ Assert.Equal("Wire Gates", (string?)json["title"]);
+ Assert.Equal("tool", (string?)json["type"]);
+ Assert.Equal("Logic gates.", (string?)json["description"]);
+
+ var again = ReadHeader(await ConvertOkAsync([unpacked], new Preset { TargetFormat = "gma", Archive = { CombineIntoOne = true } }));
+ var first = ReadHeader(gma);
+ Assert.Equal(first.Title, again.Title);
+ Assert.Equal(first.Type, again.Type);
+ Assert.Equal(first.Tags, again.Tags);
+ Assert.Equal(first.Entries.Select(e => (e.Name, e.Size, e.Crc)), again.Entries.Select(e => (e.Name, e.Size, e.Crc)));
+ }
+
+ private void SkipWithout7Zip()
+ {
+ var engine = fx.Converters.Single(c => c.Id == "archive");
+ Assert.SkipUnless(fx.Catalog.StatusOf(engine).IsAvailable, "7-Zip not found (pwsh build/fetch-engines.ps1 -Only 7zip)");
+ }
+
+ [Fact]
+ public async Task A_zip_of_an_addon_folder_without_addon_json_packs_from_inside_it()
+ {
+ SkipWithout7Zip();
+ var parent = fx.NewFolder();
+ // As addons are shared: "My Props/lua/…" with a readme beside the folder, no addon.json.
+ var wrapper = Directory.CreateDirectory(Path.Combine(parent, "download")).FullName;
+ AddonFolder(wrapper, "My Props", withJson: false);
+ File.WriteAllText(Path.Combine(wrapper, "README.txt"), "install into garrysmod/addons");
+ var zip = Path.Combine(parent, "my-props.zip");
+ ZipFile.CreateFromDirectory(wrapper, zip);
+
+ var header = ReadHeader(await ConvertOkAsync([zip], new Preset { TargetFormat = "gma" }));
+
+ Assert.Contains(header.Entries, e => e.Name == "lua/autorun/gates.lua");
+ Assert.Contains(header.Entries, e => e.Name == "models/gates/gate.mdl");
+ Assert.DoesNotContain(header.Entries, e => e.Name.Contains("props", StringComparison.Ordinal) || e.Name.EndsWith(".txt", StringComparison.Ordinal));
+ }
+
+ [Fact]
+ public void The_addon_root_is_found_through_wrapper_folders()
+ {
+ var parent = fx.NewFolder();
+ var nested = Directory.CreateDirectory(Path.Combine(parent, "a", "b", "My Addon", "materials", "x")).FullName;
+ File.WriteAllBytes(Path.Combine(nested, "y.vtf"), Noise(10, 1));
+ File.WriteAllText(Path.Combine(parent, "a", "readme.txt"), "hi");
+ Assert.Equal(Path.Combine(parent, "a", "b", "My Addon"), Gma.Root(parent));
+
+ // addon.json marks the root even without a known folder; a folder with several subfolders is the root itself.
+ var withJson = Directory.CreateDirectory(Path.Combine(fx.NewFolder(), "wrap", "addon")).FullName;
+ File.WriteAllText(Path.Combine(withJson, "addon.json"), "{}");
+ Assert.Equal(withJson, Gma.Root(Path.GetDirectoryName(withJson)!));
+ var several = fx.NewFolder();
+ Directory.CreateDirectory(Path.Combine(several, "one"));
+ Directory.CreateDirectory(Path.Combine(several, "two"));
+ Assert.Equal(several, Gma.Root(several));
+ }
+
+ [Theory]
+ [InlineData("lua/autorun/x.lua", true)]
+ [InlineData("materials/a/b/c.vtf", true)]
+ [InlineData("models/a.dx90.vtx", true)]
+ [InlineData("models/a.sw.vtx", false)]
+ [InlineData("gamemodes/sandbox/sandbox.txt", true)]
+ [InlineData("gamemodes/sandbox/sub/x.txt", false)]
+ [InlineData("data_static/x.json", true)]
+ [InlineData("readme.txt", false)]
+ [InlineData("lua/x.luac", false)]
+ public void The_whitelist_matches_gmads(string name, bool allowed) => Assert.Equal(allowed, Gma.IsAllowed(name));
+}
diff --git a/tests/Filee.Engines.Tests/GmaTests.cs b/tests/Filee.Engines.Tests/GmaTests.cs
new file mode 100644
index 000000000..ecfda3c21
--- /dev/null
+++ b/tests/Filee.Engines.Tests/GmaTests.cs
@@ -0,0 +1,172 @@
+// Garry's Mod addons (.gma, #1526): an addon unpacks into its files with every CRC checked (0, from gmad -nocrc, is not
+// checked) and an addon.json for its title, description, type and tags, as gmad reads it to pack the folder again. Names
+// that would leave the output folder and damaged addons are refused. The test addons are built here byte by byte.
+
+using System.Buffers.Binary;
+using System.IO.Compression;
+using System.Text;
+using System.Text.Json.Nodes;
+using Filee.Core.Conversion;
+using Filee.Core.Formats;
+using Filee.Core.Presets;
+using Filee.Engines.Archives;
+using static Filee.Engines.Tests.ArchiveBuilders;
+
+namespace Filee.Engines.Tests;
+
+public class GmaTests(EngineFixture fx) : IClassFixture
+{
+ private static CancellationToken Ct => TestContext.Current.CancellationToken;
+
+ private void SkipWithout7Zip()
+ {
+ var engine = fx.Converters.Single(c => c.Id == "archive");
+ Assert.SkipUnless(fx.Catalog.StatusOf(engine).IsAvailable, "7-Zip not found (pwsh build/fetch-engines.ps1 -Only 7zip)");
+ }
+
+ private async Task ConvertOkAsync(string source, string presetId)
+ {
+ var job = await fx.ConvertAsync([source], BuiltInData.CreatePresets().Single(p => p.Id == presetId));
+ Assert.True(job.State == JobState.Completed, string.Join("; ", job.Files.Select(f => $"{f.ErrorKey} {f.ErrorDetail}")));
+ return Assert.Single(job.Outputs);
+ }
+
+ private static readonly (string Name, byte[] Data)[] Files =
+ [
+ ("lua/autorun/gates.lua", "print('gates')\n"u8.ToArray()),
+ ("materials/gates/gate.vtf", Noise(300, 1)),
+ ("models/gates/gate.mdl", Noise(200, 2)),
+ ];
+
+ ///
+ /// A version 3 addon as gmad writes it: one required-content entry, , a CRC per file
+ /// (0 for each with off, as gmad -nocrc) and a CRC over the whole addon.
+ ///
+ private static byte[] BuildGma((string Name, byte[] Data)[] files, string description, bool crcs = true)
+ {
+ using var stream = new MemoryStream();
+ using (var writer = new BinaryWriter(stream, Encoding.UTF8, leaveOpen: true))
+ {
+ writer.Write("GMAD"u8);
+ writer.Write((byte)3);
+ writer.Write(0UL);
+ writer.Write(1_600_000_000UL);
+ writer.Write("base\0"u8); // one required content entry
+ writer.Write((byte)0);
+ writer.Write("Wire Gates\0"u8);
+ writer.Write(Encoding.UTF8.GetBytes(description + "\0"));
+ writer.Write("Author Name\0"u8);
+ writer.Write(1);
+ var number = 0u;
+ foreach (var (name, data) in files)
+ {
+ writer.Write(++number);
+ writer.Write(Encoding.UTF8.GetBytes(name + "\0"));
+ writer.Write((long)data.Length);
+ writer.Write(crcs ? Crc32.Compute(data) : 0u);
+ }
+ writer.Write(0u);
+ foreach (var (_, data) in files)
+ writer.Write(data);
+ }
+ var bytes = stream.ToArray();
+ return [.. bytes, .. BitConverter.GetBytes(crcs ? Crc32.Compute(bytes) : 0u)];
+ }
+
+ private const string JsonDescription = """{ "description": "Logic gates.", "type": "tool", "tags": ["fun", "build"] }""";
+
+ [Fact]
+ public void Gma_is_recognised_and_packs_and_unpacks_in_one_step()
+ {
+ Assert.Equal("gma", FormatRegistry.Detect("Addon.GMA")?.Id);
+ Assert.True(FormatRegistry.Get("gma")!.Writable);
+ var planner = fx.Catalog.CreatePlanner(["archive"]);
+ Assert.Single(planner.Plan("gma", "zip")!.Steps);
+ Assert.Single(planner.Plan("gma", FormatRegistry.Folder)!.Steps);
+ Assert.Single(planner.Plan("zip", "gma")!.Steps);
+ Assert.Single(planner.Plan(FormatRegistry.Folder, "gma")!.Steps);
+ }
+
+ [Fact]
+ public async Task An_addon_unpacks_with_an_addon_json_gmad_packs_it_from()
+ {
+ SkipWithout7Zip();
+ var path = Path.Combine(fx.NewFolder(), "wire-gates.gma");
+ File.WriteAllBytes(path, BuildGma(Files, JsonDescription));
+
+ var folder = await ConvertOkAsync(path, "extract");
+
+ foreach (var (name, data) in Files)
+ Assert.Equal(data, File.ReadAllBytes(Path.Combine(folder, name)));
+ var json = JsonNode.Parse(File.ReadAllText(Path.Combine(folder, "addon.json")))!;
+ Assert.Equal("Wire Gates", (string?)json["title"]);
+ Assert.Equal("tool", (string?)json["type"]);
+ Assert.Equal("Logic gates.", (string?)json["description"]);
+ Assert.Equal(["fun", "build"], json["tags"]!.AsArray().Select(t => (string?)t));
+ }
+
+ [Fact]
+ public async Task An_addon_without_checksums_becomes_a_zip()
+ {
+ SkipWithout7Zip();
+ var path = Path.Combine(fx.NewFolder(), "old.gma");
+ File.WriteAllBytes(path, BuildGma([("lua/a.lua", "x = 1"u8.ToArray()), ("sound/b.wav", Noise(64, 5))], "Just text.", crcs: false));
+
+ var zip = await ConvertOkAsync(path, "to-zip");
+
+ using var archive = ZipFile.OpenRead(zip);
+ Assert.Equal(["addon.json", "lua/a.lua", "sound/b.wav"],
+ archive.Entries.Where(e => !e.FullName.EndsWith('/')).Select(e => e.FullName).Order(StringComparer.Ordinal));
+ // An old addon's description is plain text, kept as it is.
+ var json = JsonNode.Parse(new StreamReader(archive.GetEntry("addon.json")!.Open()).ReadToEnd())!;
+ Assert.Equal("Just text.", (string?)json["description"]);
+ }
+
+ [Theory]
+ [InlineData("../evil.lua")]
+ [InlineData("lua/../../evil.lua")]
+ [InlineData("/etc/evil.lua")]
+ [InlineData("C:/Windows/evil.lua")]
+ public void A_name_leaving_the_output_folder_is_refused(string name)
+ {
+ var folder = fx.NewFolder();
+ var path = Path.Combine(folder, "evil.gma");
+ File.WriteAllBytes(path, BuildGma([("lua/fine.lua", "x = 1"u8.ToArray()), (name, "boom"u8.ToArray())], JsonDescription));
+ var target = Directory.CreateDirectory(Path.Combine(folder, "out")).FullName;
+
+ var error = Assert.Throws(() => Gma.Extract(path, target, Ct));
+
+ Assert.Contains("outside the target folder", error.Message);
+ Assert.False(File.Exists(Path.Combine(folder, "evil.lua")));
+ }
+
+ [Fact]
+ public void A_damaged_addon_is_refused()
+ {
+ var folder = fx.NewFolder();
+ var good = BuildGma([("lua/a.lua", "x = 1"u8.ToArray())], JsonDescription);
+
+ // Cut inside the file's content.
+ var cut = Path.Combine(folder, "cut.gma");
+ File.WriteAllBytes(cut, good[..^8]);
+ Assert.Throws(() => Gma.Extract(cut, Directory.CreateDirectory(Path.Combine(folder, "a")).FullName, Ct));
+
+ // A wrong CRC for the file: it follows the name and the 8-byte size.
+ var bad = (byte[])good.Clone();
+ var nameEnd = bad.AsSpan().IndexOf("lua/a.lua\0"u8) + "lua/a.lua\0"u8.Length;
+ BinaryPrimitives.WriteUInt32LittleEndian(bad.AsSpan(nameEnd + 8), 0xDEADBEEF);
+ var wrong = Path.Combine(folder, "wrong.gma");
+ File.WriteAllBytes(wrong, bad);
+ Assert.Throws(() => Gma.Extract(wrong, Directory.CreateDirectory(Path.Combine(folder, "b")).FullName, Ct));
+
+ // A file larger than the addon itself.
+ var huge = (byte[])good.Clone();
+ BinaryPrimitives.WriteInt64LittleEndian(huge.AsSpan(nameEnd), long.MaxValue / 2);
+ var tooLarge = Path.Combine(folder, "huge.gma");
+ File.WriteAllBytes(tooLarge, huge);
+ Assert.Throws(() => Gma.Extract(tooLarge, Directory.CreateDirectory(Path.Combine(folder, "c")).FullName, Ct));
+
+ File.WriteAllBytes(Path.Combine(folder, "not.gma"), "PK\x03\x04"u8.ToArray());
+ Assert.Throws(() => Gma.Extract(Path.Combine(folder, "not.gma"), folder, Ct));
+ }
+}
diff --git a/tests/Filee.Engines.Tests/SmdTests.cs b/tests/Filee.Engines.Tests/SmdTests.cs
new file mode 100644
index 000000000..73885ebb8
--- /dev/null
+++ b/tests/Filee.Engines.Tests/SmdTests.cs
@@ -0,0 +1,149 @@
+// Studiomdl data (.smd, #1526): an MDL becomes a reference SMD with its bones, bind pose and weights (as Crowbar
+// decompiles it: counter-clockwise, UVs from the bottom left, inches, Z up); other models become a static prop with one
+// root bone; an SMD's triangles become a model again.
+
+using System.Globalization;
+using System.Numerics;
+using Filee.Core.Conversion;
+using Filee.Core.Presets;
+using Filee.Engines.Models3D;
+using static Filee.Engines.Tests.SourceModelBuilder;
+
+namespace Filee.Engines.Tests;
+
+public class SmdTests(EngineFixture fx) : IClassFixture
+{
+ private static CancellationToken Ct => TestContext.Current.CancellationToken;
+
+ private async Task ConvertOkAsync(string source, string target)
+ {
+ var job = await fx.ConvertAsync([source], new Preset { TargetFormat = target });
+ Assert.True(job.State == JobState.Completed, string.Join("; ", job.Files.Select(f => $"{f.ErrorKey} {f.ErrorDetail}")));
+ return Assert.Single(job.Outputs);
+ }
+
+ /// The lines of a section ("nodes", "skeleton", "triangles") up to its "end".
+ private static List Section(string[] lines, string name)
+ {
+ var start = Array.IndexOf(lines, name);
+ Assert.True(start >= 0, $"no {name} section");
+ return [.. lines.Skip(start + 1).TakeWhile(l => l != "end")];
+ }
+
+ private static float[] Numbers(string line) =>
+ [.. line.Split(' ', StringSplitOptions.RemoveEmptyEntries).Select(f => float.Parse(f, CultureInfo.InvariantCulture))];
+
+ [Fact]
+ public void Smd_converts_to_and_from_the_3d_formats_in_one_step()
+ {
+ var planner = fx.Catalog.CreatePlanner(["model3d"]);
+ foreach (var (from, to) in new[] { ("mdl", "smd"), ("obj", "smd"), ("glb", "smd"), ("fbx", "smd"), ("smd", "glb"), ("smd", "stl"), ("smd", "obj") })
+ Assert.Single(planner.Plan(from, to)!.Steps);
+ }
+
+ [Fact]
+ public async Task A_model_becomes_a_reference_smd_with_its_bones_and_weights()
+ {
+ Bone[] bones =
+ [
+ new("root", -1, Vector3.Zero, Vector3.Zero),
+ new("lid", 0, new Vector3(0, 0, 5), new Vector3(0, 0, MathF.PI / 2)),
+ ];
+ var mdl = Write(fx.NewFolder(), "box", "", ["brick", "glass"], TileAndTriangle(weighted: true), bones);
+
+ var smd = await ConvertOkAsync(mdl, "smd");
+
+ var lines = File.ReadAllLines(smd);
+ Assert.Equal("version 1", lines[0]);
+ Assert.Equal(["0 \"root\" -1", "1 \"lid\" 0"], Section(lines, "nodes"));
+ var skeleton = Section(lines, "skeleton");
+ Assert.Equal("time 0", skeleton[0]);
+ Assert.Equal([1, 0, 0, 5, 0, 0, MathF.PI / 2], Numbers(skeleton[2]), new FloatComparer());
+
+ var triangles = Section(lines, "triangles");
+ Assert.Equal(3 * 4, triangles.Count); // three triangles, each a material line and three vertices
+ Assert.Equal(["brick", "brick", "glass"], new[] { triangles[0], triangles[4], triangles[8] });
+ // The tile's first triangle (0, 1, 2) in Source's order comes reversed: (10, 10, 0), (0, 10, 0), (0, 0, 0).
+ Assert.Equal([0, 10, 10, 0, 0, 0, 1, 1, 1, 1, 0, 1], Numbers(triangles[1]), new FloatComparer());
+ Assert.Equal([0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1], Numbers(triangles[3]), new FloatComparer());
+ // Its corner at (10, 0, 0) is weighted 0.75 / 0.25 to both bones; UV (1, 1) from the top is (1, 0) from the bottom.
+ Assert.Equal([0, 10, 0, 0, 0, 0, 1, 1, 0, 2, 0, 0.75f, 1, 0.25f], Numbers(triangles[5]), new FloatComparer());
+ // The triangle above it hangs on the lid.
+ Assert.Equal(1, Numbers(triangles[9])[0]);
+ Assert.Equal([1, 1, 1], Numbers(triangles[9])[^3..], new FloatComparer());
+ }
+
+ [Fact]
+ public async Task An_smd_becomes_a_model_facing_the_same_way_as_its_mdl()
+ {
+ var mdl = Write(fx.NewFolder(), "box", "", ["brick", "glass"], TileAndTriangle());
+ var fromMdl = ModelConverter.Read(mdl, "mdl", null, Ct);
+
+ var smd = await ConvertOkAsync(mdl, "smd");
+ var fromSmd = ModelConverter.Read(smd, "smd", null, Ct);
+
+ Assert.Equal(fromMdl.TriangleCount, fromSmd.TriangleCount);
+ Assert.Equal(fromMdl.Bounds(), fromSmd.Bounds());
+ Assert.Equal(["brick", "glass"], fromSmd.Materials.Select(m => m.Name));
+ foreach (var part in fromSmd.Parts)
+ {
+ for (var t = 0; t < part.TriangleCount; t++)
+ Assert.True(Vector3.Dot(part.FaceNormal(t), Vector3.UnitY) > 0.99f);
+ }
+
+ var glb = await ConvertOkAsync(smd, "glb");
+ Assert.Equal(3, ModelConverter.Read(glb, "glb", null, Ct).TriangleCount);
+ }
+
+ [Fact]
+ public void Another_model_becomes_a_static_prop_and_reads_back()
+ {
+ var model = new Model3D();
+ model.Materials.Add(new Material3D { Name = "Painted Metal" });
+ var part = new MeshPart { Name = "plate", Material = 0, TexCoords = [] };
+ part.AddVertex(new Vector3(0, 0, 0), uv: new Vector2(0, 0));
+ part.AddVertex(new Vector3(254, 0, 0), uv: new Vector2(1, 0));
+ part.AddVertex(new Vector3(0, 0, -254), uv: new Vector2(0, 1));
+ part.AddTriangle(0, 1, 2);
+ model.Parts.Add(part);
+
+ var path = Path.Combine(fx.NewFolder(), "plate.smd");
+ using (var stream = File.Create(path))
+ SmdFormat.Write(model, stream, Ct);
+
+ var lines = File.ReadAllLines(path);
+ Assert.Equal(["0 \"root\" -1"], Section(lines, "nodes"));
+ var triangles = Section(lines, "triangles");
+ Assert.Equal("Painted_Metal", triangles[0]);
+ // 254 mm along X is 10 inches; −254 mm along Z (Y up) is +10 inches along Y (Z up).
+ Assert.Equal([0, 10, 0, 0], Numbers(triangles[2])[..4], new FloatComparer());
+ Assert.Equal([0, 0, 10, 0], Numbers(triangles[3])[..4], new FloatComparer());
+ Assert.EndsWith(" 1 0 1.000000", triangles[1]);
+
+ var back = SmdFormat.Read(path, null, Ct);
+ back.Validate();
+ Assert.Equal(1, back.TriangleCount);
+ Assert.True(Vector3.Distance(model.Bounds().Max, back.Bounds().Max) < 0.01f);
+ Assert.True(Vector3.Distance(model.Bounds().Min, back.Bounds().Min) < 0.01f);
+ Assert.Equal(new Vector2(1, 0), back.Parts[0].TexCoord(1));
+ }
+
+ [Fact]
+ public void A_broken_smd_is_refused_with_its_line()
+ {
+ var path = Path.Combine(fx.NewFolder(), "broken.smd");
+ File.WriteAllText(path, "version 1\ntriangles\nmat\n0 1 2 3 0 0 1 0 0\n0 1 2 x 0 0 1 0 0\n");
+ var error = Assert.Throws(() => SmdFormat.Read(path, null, Ct));
+ Assert.Contains("Line 5", error.Message);
+
+ File.WriteAllText(path, "version 1\nnodes\n0 \"root\" -1\nend\nskeleton\ntime 0\n0 0 0 0 0 0 0\nend\n");
+ var animation = Assert.Throws(() => ModelConverter.Read(path, "smd", null, Ct));
+ Assert.Contains("no triangles", animation.Message);
+ }
+
+ private sealed class FloatComparer : IEqualityComparer
+ {
+ public bool Equals(float a, float b) => MathF.Abs(a - b) < 0.0005f;
+ public int GetHashCode(float value) => 0;
+ }
+}
diff --git a/tests/Filee.Engines.Tests/SourceDecompileTests.cs b/tests/Filee.Engines.Tests/SourceDecompileTests.cs
new file mode 100644
index 000000000..323836b8b
--- /dev/null
+++ b/tests/Filee.Engines.Tests/SourceDecompileTests.cs
@@ -0,0 +1,187 @@
+// Decompiling Source engine models (#1526): an .mdl with its .vvd, .vtx and .phy becomes a folder with the QC and SMDs
+// studiomdl builds it from: a reference SMD per body group model, an SMD for a LOD that differs, the physics SMD, and an
+// animation SMD per animation, with the QC's body groups, LODs, materials, attachments, sequences, events and collision
+// model. The test models are built by SourceModelBuilder; damaged animations and physics are refused.
+
+using System.Globalization;
+using System.Numerics;
+using Filee.Core.Conversion;
+using Filee.Core.Formats;
+using Filee.Core.Presets;
+using Filee.Engines.Models3D;
+
+namespace Filee.Engines.Tests;
+
+public class SourceDecompileTests(EngineFixture fx) : IClassFixture
+{
+ private static CancellationToken Ct => TestContext.Current.CancellationToken;
+
+ private static readonly SourceModelBuilder.Bone[] TwoBones =
+ [
+ new("root", -1, Vector3.Zero, Vector3.Zero),
+ new("arm", 0, new Vector3(0, 0, 5), new Vector3(0.5f, 0, 0)),
+ ];
+
+ private async Task DecompileAsync(string mdl)
+ {
+ var preset = BuiltInData.CreatePresets().Single(p => p.Id == BuiltInData.DecompileModelId);
+ var job = await fx.ConvertAsync([mdl], preset);
+ Assert.True(job.State == JobState.Completed, string.Join("; ", job.Files.Select(f => $"{f.ErrorKey} {f.ErrorDetail}")));
+ var folder = Assert.Single(job.Outputs);
+ Assert.True(Directory.Exists(folder));
+ return folder;
+ }
+
+ /// The numbers of an SMD line, after its first words.
+ private static float[] Numbers(string line, int skip = 1) =>
+ [.. line.Split(' ', StringSplitOptions.RemoveEmptyEntries).Skip(skip).Select(n => float.Parse(n, CultureInfo.InvariantCulture))];
+
+ [Fact]
+ public void Mdl_decompiles_into_a_folder_in_one_step()
+ {
+ var planner = fx.Catalog.CreatePlanner(["model3d"]);
+ Assert.Single(planner.Plan("mdl", FormatRegistry.Folder)!.Steps);
+ Assert.Null(planner.Plan("glb", FormatRegistry.Folder));
+ var preset = BuiltInData.CreatePresets().Single(p => p.Id == BuiltInData.DecompileModelId);
+ Assert.Equal(FormatRegistry.Folder, preset.TargetFormat);
+ }
+
+ [Fact]
+ public async Task A_model_becomes_a_qc_with_its_reference_lod_and_physics_smds()
+ {
+ var meshes = SourceModelBuilder.TileAndTriangle();
+ // LOD 1 keeps only the floor tile's first triangle.
+ SourceModelBuilder.Mesh[] lod1 = [meshes[0] with { Triangles = [0, 1, 2] }, meshes[1]];
+ var mdl = SourceModelBuilder.Write(fx.NewFolder(), "tile", "models\\props\\", ["brick", "glass"], meshes,
+ extras: new SourceModelBuilder.Extras(Lod1: lod1, Physics: true, Attachment: true), bones: TwoBones);
+
+ var folder = await DecompileAsync(mdl);
+
+ Assert.Equal(["tile.qc", "tile.smd", "tile_lod1.smd", "tile_physics.smd"],
+ Directory.GetFiles(folder).Select(f => Path.GetFileName(f)).Order(StringComparer.Ordinal));
+ var qc = File.ReadAllText(Path.Combine(folder, "tile.qc"));
+ Assert.Contains("$modelname \"test/tile.mdl\"", qc);
+ Assert.Contains("$bodygroup \"body\"\n{\n\tstudio \"tile.smd\"\n}", qc);
+ Assert.Contains("$lod 50\n{\n\treplacemodel \"tile.smd\" \"tile_lod1.smd\"\n}", qc);
+ Assert.Contains("$cdmaterials \"models\\props\\\"", qc);
+ Assert.Contains("$attachment \"muzzle\" \"arm\" 1 2 3 rotate 0 0 0", qc);
+ Assert.Contains("$collisionmodel \"tile_physics.smd\"\n{\n\t$mass 5\n\t$inertia 1\n\t$damping 0\n\t$rotdamping 0\n}", qc);
+ // What Crowbar writes for every model: contents, both boxes and a $definebone per bone (pitch, yaw, roll in degrees).
+ Assert.Contains("$contents \"solid\"\n", qc);
+ Assert.Contains("$cbox 0 0 0 0 0 0\n", qc);
+ Assert.Contains("$definebone \"root\" \"\" 0 0 0 0 0 0 0 0 0 0 0 0\n", qc);
+ Assert.Contains("$definebone \"arm\" \"root\" 0 0 5 0 0 28.64", qc);
+
+ // The reference SMD: both bones, then 3 triangles (2 + 1), counter-clockwise as SMDs are.
+ var reference = File.ReadAllLines(Path.Combine(folder, "tile.smd"));
+ Assert.Equal(["version 1", "nodes", "0 \"root\" -1", "1 \"arm\" 0", "end", "skeleton", "time 0"], reference[..7]);
+ Assert.Equal(3, reference.Count(l => l is "brick" or "glass"));
+ Assert.Equal(1, File.ReadAllLines(Path.Combine(folder, "tile_lod1.smd")).Count(l => l == "brick"));
+
+ // The physics SMD: the tetrahedron's four faces, its corners in inches.
+ var physics = File.ReadAllLines(Path.Combine(folder, "tile_physics.smd"));
+ var start = Array.IndexOf(physics, "triangles");
+ Assert.Equal(4, physics.Count(l => l == "phy"));
+ var corners = physics.Skip(start + 1).Where(l => l is not ("phy" or "end"))
+ .Select(l => Numbers(l)[..3]).Select(n => new Vector3(n[0], n[1], n[2])).Distinct().ToList();
+ Assert.Equal(4, corners.Count);
+ foreach (var corner in new[] { Vector3.Zero, Vector3.UnitX, Vector3.UnitY, Vector3.UnitZ })
+ Assert.Contains(corners, c => Vector3.Distance(c, corner) < 1e-4f);
+ }
+
+ [Fact]
+ public async Task Animations_become_animation_smds_and_sequences()
+ {
+ var mdl = SourceModelBuilder.Write(fx.NewFolder(), "waver", "models\\", ["brick", "glass"], SourceModelBuilder.TileAndTriangle(),
+ extras: new SourceModelBuilder.Extras(Animation: true), bones: TwoBones);
+
+ var folder = await DecompileAsync(mdl);
+
+ var qc = File.ReadAllText(Path.Combine(folder, "waver.qc"));
+ // As Crowbar writes it: the brace on the line, a backslash in the path, the fade times always.
+ Assert.Contains("$sequence \"wave\" {\n\t\"waver_anims\\wave.smd\"\n\tactivity \"ACT_IDLE\" 1\n\tfadein 0.2\n\tfadeout 0.2\n"
+ + "\tfps 30\n\tloop\n\t{ event 5004 1 \"test.sound\" }\n}", qc);
+ var lines = File.ReadAllLines(Path.Combine(folder, "waver_anims", "wave.smd"));
+ Assert.Equal(3, lines.Count(l => l.StartsWith("time ", StringComparison.Ordinal)));
+ for (var frame = 0; frame < 3; frame++)
+ {
+ var at = Array.IndexOf(lines, $"time {frame}");
+ // Bone 0: the raw rotation (none) and position (1, 2, 3).
+ Assert.Equal(new float[] { 1, 2, 3, 0, 0, 0 }, Numbers(lines[at + 1]));
+ // Bone 1: its bind position, and its X rotation 0.5 + one radian a frame (100 × 0.01).
+ var arm = Numbers(lines[at + 2]);
+ Assert.Equal(new float[] { 0, 0, 5 }, arm[..3]);
+ Assert.Equal(0.5f + frame, arm[3], 4);
+ }
+ }
+
+ [Fact]
+ public async Task Illumination_and_eye_positions_are_turned_back_as_the_qc_wrote_them()
+ {
+ var folder = fx.NewFolder();
+ var mdl = SourceModelBuilder.Write(folder, "lit", "models\\", ["brick", "glass"], SourceModelBuilder.TileAndTriangle());
+ var bytes = File.ReadAllBytes(mdl);
+ // studiomdl stores QC (x, y, z) as (−y, x, z): (2, −1, 3) stored was (−1, −2, 3) in the QC.
+ foreach (var (at, value) in new[] { (80, 4f), (84, 5f), (88, 6f), (92, 2f), (96, -1f), (100, 3f) })
+ BitConverter.GetBytes(value).CopyTo(bytes, at);
+ File.WriteAllBytes(mdl, bytes);
+
+ var qc = File.ReadAllText(Path.Combine(await DecompileAsync(mdl), "lit.qc"));
+
+ Assert.Contains("$illumposition -1 -2 3\n", qc);
+ Assert.Contains("$eyeposition 5 -4 6\n", qc);
+ }
+
+ [Fact]
+ public void Euler_angles_survive_a_quaternion()
+ {
+ var random = new Random(1526);
+ for (var i = 0; i < 1000; i++)
+ {
+ var angles = new Vector3(random.NextSingle() * 6 - 3, random.NextSingle() * 3 - 1.5f, random.NextSingle() * 6 - 3);
+ var q = SourceAnimation.FromEuler(angles);
+ var back = SourceAnimation.FromEuler(SourceAnimation.ToEuler(q));
+ Assert.True(MathF.Abs(Quaternion.Dot(q, back)) > 0.99999f, $"{angles} came back as {SourceAnimation.ToEuler(q)}");
+ }
+ // Source's order: yaw (Z) applied last; a quarter turn about Z takes X to Y.
+ var turn = SourceAnimation.FromEuler(new Vector3(0, 0, MathF.PI / 2));
+ Assert.True(Vector3.Distance(Vector3.UnitY, Vector3.Transform(Vector3.UnitX, turn)) < 1e-5f);
+ }
+
+ [Fact]
+ public async Task A_model_with_a_damaged_animation_or_physics_is_refused()
+ {
+ var folder = fx.NewFolder();
+ var mdl = SourceModelBuilder.Write(folder, "bad", "models\\", ["brick", "glass"], SourceModelBuilder.TileAndTriangle(),
+ extras: new SourceModelBuilder.Extras(Animation: true, Physics: true), bones: TwoBones);
+ var original = File.ReadAllBytes(mdl);
+
+ // An animation claiming a million frames.
+ var bytes = (byte[])original.Clone();
+ var description = BitConverter.ToInt32(bytes, 184);
+ BitConverter.GetBytes(1_000_000).CopyTo(bytes, description + 16);
+ File.WriteAllBytes(mdl, bytes);
+ Assert.Throws(() => SourceDecompiler.Decompile(mdl, Directory.CreateDirectory(Path.Combine(folder, "a")).FullName, Ct));
+
+ // A bone entry chaining back to itself.
+ bytes = (byte[])original.Clone();
+ var animation = description + BitConverter.ToInt32(bytes, description + 56);
+ var second = animation + BitConverter.ToInt16(bytes, animation + 2);
+ BitConverter.GetBytes((short)0).CopyTo(bytes, second + 2);
+ BitConverter.GetBytes((short)(animation - second)).CopyTo(bytes, second + 2);
+ File.WriteAllBytes(mdl, bytes);
+ Assert.Throws(() => SourceDecompiler.Decompile(mdl, Directory.CreateDirectory(Path.Combine(folder, "b")).FullName, Ct));
+
+ // A collision tree whose node points at itself.
+ File.WriteAllBytes(mdl, original);
+ var phy = Path.ChangeExtension(mdl, ".phy");
+ var physics = File.ReadAllBytes(phy);
+ var surface = 16 + 4 + 28;
+ var node = surface + BitConverter.ToInt32(physics, surface + 32);
+ BitConverter.GetBytes(-28).CopyTo(physics, node);
+ File.WriteAllBytes(phy, physics);
+ var error = await Assert.ThrowsAsync(() => Task.Run(() =>
+ SourceDecompiler.Decompile(mdl, Directory.CreateDirectory(Path.Combine(folder, "c")).FullName, Ct), Ct));
+ Assert.Contains("damaged", error.Message);
+ }
+}
diff --git a/tests/Filee.Engines.Tests/SourceModelBuilder.cs b/tests/Filee.Engines.Tests/SourceModelBuilder.cs
new file mode 100644
index 000000000..e85f18749
--- /dev/null
+++ b/tests/Filee.Engines.Tests/SourceModelBuilder.cs
@@ -0,0 +1,421 @@
+// Builds Source engine models (.mdl version 48 with its .vvd and .dx90.vtx) byte by byte for the tests: bones, textures
+// with one texture folder, one body part with one model, and one mesh per material, each a triangle list in one strip
+// group. Layouts as in Valve's studio.h and optimize.h (see Models3D/SourceModel.cs). For decompiling (#1526) a model
+// can also get an attachment, a second LOD, one animation with its sequence and event, and a .phy with one convex piece.
+
+using System.Buffers.Binary;
+using System.Numerics;
+using System.Text;
+
+namespace Filee.Engines.Tests;
+
+internal static class SourceModelBuilder
+{
+ /// A vertex in Source's axes and units (inches, Z up), UV from the top left, weighted to up to two bones.
+ public sealed record Vertex(Vector3 Position, Vector3 Normal, Vector2 Uv, int Bone = 0, int Bone2 = -1, float Weight2 = 0);
+
+ /// A mesh: its material (an index into the textures), vertices and triangles in Source's (clockwise) order.
+ public sealed record Mesh(int Material, Vertex[] Vertices, int[] Triangles);
+
+ /// A bone in its bind pose relative to its parent.
+ public sealed record Bone(string Name, int Parent, Vector3 Position, Vector3 Rotation);
+
+ public const int Checksum = 0x1EE7;
+
+ ///
+ /// The extras a decompiled model is tested with: adds a three-frame animation "@wave"
+ /// (bone 0 at rest with a Quaternion64 rotation and a Vector48 position of (1, 2, 3); bone 1's X rotation going up 1
+ /// radian a frame from its bind pose, in a run-length encoded track) and a looping sequence "wave" with activity
+ /// ACT_IDLE and event 5004 "test.sound" halfway; adds "muzzle" on bone 1 at (1, 2, 3);
+ /// is a second LOD (switch point 50) with these triangles instead;
+ /// writes a .phy with one convex piece, a tetrahedron with corners (0, 0, 0), (1, 0, 0), (0, 1, 0) and (0, 0, 1).
+ ///
+ public sealed record Extras(bool Animation = false, bool Attachment = false, Mesh[]? Lod1 = null, bool Physics = false);
+
+ ///
+ /// Writes ".mdl", ".vvd" and ".dx90.vtx" into and returns the .mdl;
+ /// sets a different checksum in the .vvd; stores LOD 0 as
+ /// two runs of vertices.
+ ///
+ public static string Write(string folder, string name, string textureFolder, string[] textures, Mesh[] meshes,
+ Bone[]? bones = null, int vvdChecksum = Checksum, bool fixups = false, Extras? extras = null)
+ {
+ bones ??= [new Bone("static_prop", -1, Vector3.Zero, Vector3.Zero)];
+ extras ??= new Extras();
+ Directory.CreateDirectory(folder);
+ var path = Path.Combine(folder, name + ".mdl");
+ File.WriteAllBytes(path, Mdl(name, textureFolder, textures, meshes, bones, extras));
+ File.WriteAllBytes(Path.Combine(folder, name + ".vvd"), Vvd(meshes, vvdChecksum, fixups));
+ File.WriteAllBytes(Path.Combine(folder, name + ".dx90.vtx"), Vtx(meshes, extras.Lod1));
+ if (extras.Physics)
+ File.WriteAllBytes(Path.Combine(folder, name + ".phy"), Phy(name));
+ return path;
+ }
+
+ private static byte[] Mdl(string name, string textureFolder, string[] textures, Mesh[] meshes, Bone[] bones, Extras extras)
+ {
+ const int header = 408, bone = 216, texture = 64, bodyPart = 16, model = 148, mesh = 116;
+ var boneAt = header;
+ var textureAt = boneAt + bones.Length * bone;
+ var folderTableAt = textureAt + textures.Length * texture;
+ var skinAt = folderTableAt + 4;
+ var bodyPartAt = skinAt + textures.Length * 2 + 2;
+ var modelAt = bodyPartAt + bodyPart;
+ var meshAt = modelAt + model;
+ var stringsAt = meshAt + meshes.Length * mesh;
+
+ var strings = new MemoryStream();
+ int Str(string text)
+ {
+ var at = stringsAt + (int)strings.Length;
+ strings.Write(Encoding.ASCII.GetBytes(text + "\0"));
+ return at;
+ }
+
+ var data = new byte[stringsAt];
+ var span = data.AsSpan();
+ "IDST"u8.CopyTo(span);
+ I32(span, 4, 48);
+ I32(span, 8, Checksum);
+ Encoding.ASCII.GetBytes($"test/{name}.mdl").CopyTo(span[12..]);
+ I32(span, 156, bones.Length);
+ I32(span, 332, 1); // CONTENTS_SOLID, as studiomdl writes by default
+ I32(span, 160, boneAt);
+ I32(span, 204, textures.Length);
+ I32(span, 208, textureAt);
+ I32(span, 212, 1);
+ I32(span, 216, folderTableAt);
+ I32(span, 220, textures.Length);
+ I32(span, 224, 1);
+ I32(span, 228, skinAt);
+ I32(span, 232, 1);
+ I32(span, 236, bodyPartAt);
+
+ for (var i = 0; i < bones.Length; i++)
+ {
+ var at = boneAt + i * bone;
+ I32(span, at, Str(bones[i].Name) - at);
+ I32(span, at + 4, bones[i].Parent);
+ Vec(span, at + 32, bones[i].Position);
+ F32(span, at + 56, 1); // quaternion w
+ Vec(span, at + 60, bones[i].Rotation);
+ }
+ for (var i = 0; i < textures.Length; i++)
+ {
+ var at = textureAt + i * texture;
+ I32(span, at, Str(textures[i]) - at);
+ BinaryPrimitives.WriteInt16LittleEndian(span[(skinAt + i * 2)..], (short)i);
+ }
+ I32(span, folderTableAt, Str(textureFolder));
+ I32(span, bodyPartAt, Str("body") - bodyPartAt);
+ I32(span, bodyPartAt + 4, 1);
+ I32(span, bodyPartAt + 8, 1);
+ I32(span, bodyPartAt + 12, modelAt - bodyPartAt);
+ Encoding.ASCII.GetBytes(name).CopyTo(span[modelAt..]);
+ I32(span, modelAt + 72, meshes.Length);
+ I32(span, modelAt + 76, meshAt - modelAt);
+ I32(span, modelAt + 80, meshes.Sum(m => m.Vertices.Length));
+ I32(span, modelAt + 84, 0);
+ var first = 0;
+ for (var i = 0; i < meshes.Length; i++)
+ {
+ var at = meshAt + i * mesh;
+ I32(span, at, meshes[i].Material);
+ I32(span, at + 4, modelAt - at);
+ I32(span, at + 8, meshes[i].Vertices.Length);
+ I32(span, at + 12, first);
+ first += meshes[i].Vertices.Length;
+ }
+ if (extras.Animation && bones.Length > 1)
+ Vec(span, boneAt + 216 + 84, new Vector3(0.01f, 0, 0)); // bone 1's rotation scale
+ var result = data.Concat(strings.ToArray()).ToArray();
+ if (extras.Animation || extras.Attachment)
+ result = [.. result, .. MdlExtras(result, extras)];
+ I32(result, 76, result.Length);
+ return result;
+ }
+
+ /// The attachment, animation and sequence of , appended to (whose header is set).
+ private static byte[] MdlExtras(byte[] mdl, Extras extras)
+ {
+ const int attachment = 92, description = 100, sequence = 212, animationEvent = 80;
+ var at = mdl.Length;
+ var attachmentAt = at;
+ var descriptionAt = attachmentAt + (extras.Attachment ? attachment : 0);
+ var animationAt = descriptionAt + description;
+ // Bone 0: Quaternion64 (identity) and Vector48 (1, 2, 3); bone 1: three track offsets, then the X track.
+ var entry1 = animationAt + 4 + 8 + 6;
+ var trackAt = entry1 + 4 + 6;
+ var sequenceAt = trackAt + 2 + 3 * 2;
+ var eventAt = sequenceAt + sequence;
+ var indexAt = eventAt + animationEvent;
+ var stringsAt = indexAt + 4;
+
+ var strings = new MemoryStream();
+ int Str(string text)
+ {
+ var position = stringsAt + (int)strings.Length;
+ strings.Write(Encoding.ASCII.GetBytes(text + "\0"));
+ return position;
+ }
+ var data = new byte[stringsAt - at];
+ Span Local(int absolute) => data.AsSpan(absolute - at);
+
+ if (extras.Attachment)
+ {
+ I32(mdl, 240, 1);
+ I32(mdl, 244, attachmentAt);
+ I32(Local(attachmentAt), 0, Str("muzzle") - attachmentAt);
+ I32(Local(attachmentAt), 8, 1);
+ // Identity rotation, position (1, 2, 3): rows of a 3×4 matrix.
+ F32(Local(attachmentAt), 12, 1);
+ F32(Local(attachmentAt), 24, 1);
+ F32(Local(attachmentAt), 32, 1);
+ F32(Local(attachmentAt), 40, 2);
+ F32(Local(attachmentAt), 52, 1);
+ F32(Local(attachmentAt), 56, 3);
+ }
+ if (extras.Animation)
+ {
+ I32(mdl, 180, 1);
+ I32(mdl, 184, descriptionAt);
+ var d = Local(descriptionAt);
+ I32(d, 4, Str("@wave") - descriptionAt);
+ F32(d, 8, 30);
+ I32(d, 16, 3);
+ I32(d, 56, animationAt - descriptionAt);
+
+ var a = Local(animationAt);
+ a[0] = 0;
+ a[1] = 0x21; // Quaternion64 rotation and Vector48 position
+ BinaryPrimitives.WriteInt16LittleEndian(a[2..], (short)(entry1 - animationAt));
+ const ulong zero = 1048576;
+ BinaryPrimitives.WriteUInt64LittleEndian(a[4..], zero | (zero << 21) | (zero << 42));
+ BinaryPrimitives.WriteHalfLittleEndian(a[12..], (Half)1);
+ BinaryPrimitives.WriteHalfLittleEndian(a[14..], (Half)2);
+ BinaryPrimitives.WriteHalfLittleEndian(a[16..], (Half)3);
+ var e1 = Local(entry1);
+ e1[0] = 1;
+ e1[1] = 0x08; // animated rotation
+ BinaryPrimitives.WriteInt16LittleEndian(e1[4..], (short)(trackAt - (entry1 + 4)));
+ var track = Local(trackAt);
+ track[0] = 3; // valid
+ track[1] = 3; // total
+ BinaryPrimitives.WriteInt16LittleEndian(track[2..], 0);
+ BinaryPrimitives.WriteInt16LittleEndian(track[4..], 100);
+ BinaryPrimitives.WriteInt16LittleEndian(track[6..], 200);
+
+ I32(mdl, 188, 1);
+ I32(mdl, 192, sequenceAt);
+ var q = Local(sequenceAt);
+ I32(q, 4, Str("wave") - sequenceAt);
+ I32(q, 8, Str("ACT_IDLE") - sequenceAt);
+ I32(q, 12, 1); // looping
+ I32(q, 20, 1);
+ I32(q, 24, 1);
+ I32(q, 28, eventAt - sequenceAt);
+ I32(q, 56, 1);
+ I32(q, 60, indexAt - sequenceAt);
+ I32(q, 68, 1);
+ I32(q, 72, 1);
+ I32(q, 76, -1);
+ I32(q, 80, -1);
+ F32(q, 104, 0.2f);
+ F32(q, 108, 0.2f);
+ var e = Local(eventAt);
+ F32(e, 0, 0.5f);
+ I32(e, 4, 5004);
+ Encoding.ASCII.GetBytes("test.sound").CopyTo(e[12..]);
+ }
+ return [.. data, .. strings.ToArray()];
+ }
+
+ /// A .phy with one solid of one convex piece (see ) and its key values.
+ private static byte[] Phy(string name)
+ {
+ // Source (x, y, z) is IVP (x, −z, y) in metres.
+ Vector3[] corners = [Vector3.Zero, Vector3.UnitX, Vector3.UnitY, Vector3.UnitZ];
+ int[][] faces = [[0, 2, 1], [0, 1, 3], [0, 3, 2], [1, 2, 3]];
+ const int header = 16, vphy = 28, surface = 48, ledge = 16, triangle = 16, point = 16, node = 28;
+ var solidAt = header + 4;
+ var surfaceAt = solidAt + vphy;
+ var ledgeAt = surfaceAt + surface;
+ var pointsAt = ledgeAt + ledge + faces.Length * triangle;
+ var nodeAt = pointsAt + corners.Length * point;
+ var end = nodeAt + node;
+ var data = new byte[end];
+ var span = data.AsSpan();
+ I32(span, 0, header);
+ I32(span, 8, 1);
+ I32(span, 12, Checksum);
+ I32(span, header, end - solidAt);
+ "VPHY"u8.CopyTo(span[solidAt..]);
+ I32(span, surfaceAt + 32, nodeAt - surfaceAt);
+ "IVPS"u8.CopyTo(span[(surfaceAt + 44)..]);
+ I32(span, ledgeAt, pointsAt - ledgeAt);
+ BinaryPrimitives.WriteInt16LittleEndian(span[(ledgeAt + 12)..], (short)faces.Length);
+ for (var f = 0; f < faces.Length; f++)
+ {
+ for (var e = 0; e < 3; e++)
+ BinaryPrimitives.WriteUInt16LittleEndian(span[(ledgeAt + ledge + f * triangle + 4 + e * 4)..], (ushort)faces[f][e]);
+ }
+ for (var c = 0; c < corners.Length; c++)
+ Vec(span, pointsAt + c * point, new Vector3(corners[c].X, -corners[c].Z, corners[c].Y) * 0.0254f);
+ I32(span, nodeAt + 4, ledgeAt - nodeAt);
+ var text = $"solid {{\n\"index\" \"0\"\n\"name\" \"{name}_phys\"\n\"mass\" \"5.000000\"\n\"inertia\" \"1.000000\"\n"
+ + "\"damping\" \"0.000000\"\n\"rotdamping\" \"0.000000\"\n}\neditparams {\n\"rootname\" \"\"\n\"totalmass\" \"5.000000\"\n\"concave\" \"0\"\n}\n";
+ return [.. data, .. Encoding.ASCII.GetBytes(text + "\0")];
+ }
+
+ private static byte[] Vvd(Mesh[] meshes, int checksum, bool fixups)
+ {
+ var vertices = meshes.SelectMany(m => m.Vertices).ToArray();
+ var fixupCount = fixups ? 2 : 0;
+ var fixupAt = 64;
+ var vertexAt = fixupAt + fixupCount * 12;
+ var data = new byte[vertexAt + vertices.Length * 48];
+ var span = data.AsSpan();
+ "IDSV"u8.CopyTo(span);
+ I32(span, 4, 4);
+ I32(span, 8, checksum);
+ I32(span, 12, 1);
+ I32(span, 16, vertices.Length);
+ I32(span, 48, fixupCount);
+ I32(span, 52, fixupAt);
+ I32(span, 56, vertexAt);
+ I32(span, 60, data.Length);
+ if (fixups)
+ {
+ var half = vertices.Length / 2;
+ I32(span, fixupAt, 0);
+ I32(span, fixupAt + 4, 0);
+ I32(span, fixupAt + 8, half);
+ I32(span, fixupAt + 12, 0);
+ I32(span, fixupAt + 16, half);
+ I32(span, fixupAt + 20, vertices.Length - half);
+ }
+ for (var i = 0; i < vertices.Length; i++)
+ {
+ var at = vertexAt + i * 48;
+ var v = vertices[i];
+ var two = v.Bone2 >= 0;
+ F32(span, at, two ? 1 - v.Weight2 : 1);
+ F32(span, at + 4, two ? v.Weight2 : 0);
+ span[at + 12] = (byte)v.Bone;
+ span[at + 13] = (byte)Math.Max(0, v.Bone2);
+ span[at + 15] = (byte)(two ? 2 : 1);
+ Vec(span, at + 16, v.Position);
+ Vec(span, at + 28, v.Normal);
+ F32(span, at + 40, v.Uv.X);
+ F32(span, at + 44, v.Uv.Y);
+ }
+ return data;
+ }
+
+ private static byte[] Vtx(Mesh[] meshes, Mesh[]? lod1)
+ {
+ const int header = 36, bodyPart = 8, model = 8, lod = 12, mesh = 9, group = 25, strip = 27;
+ Mesh[][] lods = lod1 is null ? [meshes] : [meshes, lod1];
+ var bodyPartAt = header;
+ var modelAt = bodyPartAt + bodyPart;
+ var lodAt = modelAt + model;
+ var meshAt = lodAt + lods.Length * lod;
+ var groupAt = meshAt + lods.Sum(l => l.Length) * mesh;
+ var dataAt = groupAt + lods.Sum(l => l.Length) * group;
+
+ var tail = new MemoryStream();
+ var bytes = new byte[dataAt];
+ var span = bytes.AsSpan();
+ I32(span, 0, 7);
+ I32(span, 4, 24);
+ I32(span, 16, Checksum);
+ I32(span, 20, lods.Length);
+ I32(span, 28, 1);
+ I32(span, 32, bodyPartAt);
+ I32(span, bodyPartAt, 1);
+ I32(span, bodyPartAt + 4, modelAt - bodyPartAt);
+ I32(span, modelAt, lods.Length);
+ I32(span, modelAt + 4, lodAt - modelAt);
+ var meshIndex = 0;
+ for (var l = 0; l < lods.Length; l++)
+ {
+ var lodPos = lodAt + l * lod;
+ I32(span, lodPos, lods[l].Length);
+ I32(span, lodPos + 4, meshAt + meshIndex * mesh - lodPos);
+ F32(span, lodPos + 8, l * 50);
+ meshIndex += lods[l].Length;
+ }
+ var all = lods.SelectMany(l => l).ToArray();
+ for (var i = 0; i < all.Length; i++)
+ {
+ var m = all[i];
+ var meshPos = meshAt + i * mesh;
+ var groupPos = groupAt + i * group;
+ I32(span, meshPos, 1);
+ I32(span, meshPos + 4, groupPos - meshPos);
+
+ var vertexPos = dataAt + (int)tail.Length;
+ foreach (var index in Enumerable.Range(0, m.Vertices.Length))
+ {
+ var vertex = new byte[9];
+ vertex[3] = 1;
+ BinaryPrimitives.WriteUInt16LittleEndian(vertex.AsSpan(4), (ushort)index);
+ tail.Write(vertex);
+ }
+ var indexPos = dataAt + (int)tail.Length;
+ var u16 = new byte[2];
+ foreach (var index in m.Triangles)
+ {
+ BinaryPrimitives.WriteUInt16LittleEndian(u16, (ushort)index);
+ tail.Write(u16);
+ }
+ var stripPos = dataAt + (int)tail.Length;
+ var stripBytes = new byte[strip];
+ I32(stripBytes, 0, m.Triangles.Length);
+ I32(stripBytes, 8, m.Vertices.Length);
+ BinaryPrimitives.WriteInt16LittleEndian(stripBytes.AsSpan(16), 1);
+ stripBytes[18] = 0x01; // triangle list
+ tail.Write(stripBytes);
+
+ I32(span, groupPos, m.Vertices.Length);
+ I32(span, groupPos + 4, vertexPos - groupPos);
+ I32(span, groupPos + 8, m.Triangles.Length);
+ I32(span, groupPos + 12, indexPos - groupPos);
+ I32(span, groupPos + 16, 1);
+ I32(span, groupPos + 20, stripPos - groupPos);
+ }
+ return [.. bytes, .. tail.ToArray()];
+ }
+
+ private static void I32(Span span, int at, int value) => BinaryPrimitives.WriteInt32LittleEndian(span[at..], value);
+
+ private static void F32(Span span, int at, float value) => BinaryPrimitives.WriteSingleLittleEndian(span[at..], value);
+
+ private static void Vec(Span span, int at, Vector3 v)
+ {
+ F32(span, at, v.X);
+ F32(span, at + 4, v.Y);
+ F32(span, at + 8, v.Z);
+ }
+
+ ///
+ /// A 10 × 10 inch floor tile facing up (material 0, four vertices, two triangles) and a triangle 5 inches above it
+ /// (material 1), both clockwise as Source stores them.
+ ///
+ public static Mesh[] TileAndTriangle(bool weighted = false) =>
+ [
+ new(0,
+ [
+ new(new(0, 0, 0), Vector3.UnitZ, new(0, 1)),
+ new(new(0, 10, 0), Vector3.UnitZ, new(0, 0)),
+ new(new(10, 10, 0), Vector3.UnitZ, new(1, 0)),
+ new(new(10, 0, 0), Vector3.UnitZ, new(1, 1), weighted ? 0 : 0, weighted ? 1 : -1, weighted ? 0.25f : 0),
+ ], [0, 1, 2, 0, 2, 3]),
+ new(1,
+ [
+ new(new(0, 0, 5), Vector3.UnitZ, new(0, 0), weighted ? 1 : 0),
+ new(new(0, 10, 5), Vector3.UnitZ, new(0, 1), weighted ? 1 : 0),
+ new(new(10, 10, 5), Vector3.UnitZ, new(1, 1), weighted ? 1 : 0),
+ ], [0, 1, 2]),
+ ];
+}
diff --git a/tests/Filee.Engines.Tests/SourceModelTests.cs b/tests/Filee.Engines.Tests/SourceModelTests.cs
new file mode 100644
index 000000000..61c811ca7
--- /dev/null
+++ b/tests/Filee.Engines.Tests/SourceModelTests.cs
@@ -0,0 +1,159 @@
+// Source engine models (.mdl with .vvd and .dx90.vtx, #1526): read in their bind pose into Filee's 3D model (Y up,
+// millimetres, counter-clockwise), with their textures found through the .vmt files in the "materials" folder above them
+// and decoded from VTF. The test models are built by SourceModelBuilder.
+
+using System.Numerics;
+using Filee.Core.Conversion;
+using Filee.Core.Formats;
+using Filee.Core.Presets;
+using Filee.Engines.Magick;
+using Filee.Engines.Models3D;
+using ImageMagick;
+
+namespace Filee.Engines.Tests;
+
+public class SourceModelTests(EngineFixture fx) : IClassFixture
+{
+ private static CancellationToken Ct => TestContext.Current.CancellationToken;
+
+ private async Task> ConvertOkAsync(string source, string target)
+ {
+ var job = await fx.ConvertAsync([source], new Preset { TargetFormat = target });
+ Assert.True(job.State == JobState.Completed, string.Join("; ", job.Files.Select(f => $"{f.ErrorKey} {f.ErrorDetail}")));
+ return [.. job.Outputs];
+ }
+
+ ///
+ /// An addon folder as Garry's Mod mounts it: models/props/tile.mdl (two materials, "brick" and "glass"), the brick's
+ /// material (in a differently cased folder) pointing at a red 16 × 16 VTF, and no material for the glass.
+ ///
+ private string Addon(bool fixups = false)
+ {
+ var root = fx.NewFolder();
+ var mdl = SourceModelBuilder.Write(Path.Combine(root, "models", "props"), "tile", "models\\props\\", ["brick", "glass"],
+ SourceModelBuilder.TileAndTriangle(), fixups: fixups);
+ var materials = Directory.CreateDirectory(Path.Combine(root, "materials", "Models", "Props")).FullName;
+ File.WriteAllText(Path.Combine(materials, "brick.vmt"), """
+ // a comment
+ "VertexLitGeneric"
+ {
+ "$basetexture" "models\props\brick_color"
+ "$nocull" "1"
+ }
+ """);
+ using var red = new MagickImage(MagickColors.Red, 16, 16);
+ VtfWriter.Write([red], Path.Combine(materials, "brick_color.vtf"));
+ return mdl;
+ }
+
+ [Fact]
+ public void Mdl_is_a_3d_model_that_converts_in_one_step()
+ {
+ Assert.Equal("mdl", FormatRegistry.Detect("Tile.MDL")?.Id);
+ var planner = fx.Catalog.CreatePlanner(["model3d"]);
+ foreach (var target in new[] { "obj", "glb", "stl", "3mf", "ply", "gltf" })
+ Assert.Single(planner.Plan("mdl", target)!.Steps);
+ }
+
+ [Theory]
+ [InlineData(false)]
+ [InlineData(true)]
+ public void A_model_is_read_in_filees_axes_units_and_winding(bool fixups)
+ {
+ var model = ModelConverter.Read(Addon(fixups), "mdl", null, Ct);
+
+ Assert.Equal(2, model.Parts.Count);
+ Assert.Equal(3, model.TriangleCount);
+ Assert.Equal(7, model.VertexCount);
+ // 10 inches is 254 mm; Source's Z up becomes Y up, its Y becomes −Z.
+ var (min, max) = model.Bounds();
+ Assert.Equal(new Vector3(0, 0, -254), min, new Vector3Comparer());
+ Assert.Equal(new Vector3(254, 127, 0), max, new Vector3Comparer());
+ foreach (var part in model.Parts)
+ {
+ // Counter-clockwise: every face points the way its vertex normals (up) do.
+ for (var t = 0; t < part.TriangleCount; t++)
+ Assert.True(Vector3.Dot(part.FaceNormal(t), Vector3.UnitY) > 0.99f);
+ Assert.All(Enumerable.Range(0, part.VertexCount), v => Assert.Equal(Vector3.UnitY, part.Normal(v), new Vector3Comparer()));
+ }
+ // UVs from the bottom left: the tile's corner at the origin had v = 1 from the top.
+ var tile = model.Parts[0];
+ var origin = Enumerable.Range(0, tile.VertexCount).Single(v => tile.Position(v) == Vector3.Zero);
+ Assert.Equal(Vector2.Zero, tile.TexCoord(origin));
+ }
+
+ [Fact]
+ public void Its_textures_are_found_through_the_materials()
+ {
+ var model = ModelConverter.Read(Addon(), "mdl", null, Ct);
+
+ var brick = model.Materials.Single(m => m.Name == "brick");
+ Assert.True(brick.DoubleSided);
+ var texture = Assert.IsType(brick.BaseColorTexture);
+ Assert.Equal("png", texture.Extension);
+ using var image = new MagickImage(texture.Data);
+ Assert.Equal((16u, 16u), (image.Width, image.Height));
+ using var pixels = image.GetPixels();
+ var pixel = pixels.GetPixel(8, 8)!.ToColor()!;
+ Assert.InRange(pixel.R, 240, 255);
+ Assert.InRange(pixel.G, 0, 15);
+
+ // The game's own texture (no material in the addon) leaves the material plain.
+ Assert.Null(model.Materials.Single(m => m.Name == "glass").BaseColorTexture);
+ }
+
+ [Fact]
+ public async Task A_model_becomes_an_obj_with_its_texture_and_a_glb()
+ {
+ var mdl = Addon();
+
+ var obj = await ConvertOkAsync(mdl, "obj");
+ Assert.Contains(obj, p => p.EndsWith(".obj", StringComparison.Ordinal));
+ Assert.Contains(obj, p => p.EndsWith(".mtl", StringComparison.Ordinal));
+ Assert.Contains(obj, p => p.EndsWith(".png", StringComparison.Ordinal));
+
+ var glb = Assert.Single(await ConvertOkAsync(mdl, "glb"));
+ var back = ModelConverter.Read(glb, "glb", null, Ct);
+ Assert.Equal(3, back.TriangleCount);
+ Assert.Contains(back.Materials, m => m.BaseColorTexture is not null);
+ }
+
+ [Fact]
+ public void A_model_without_its_vvd_or_with_another_builds_vvd_is_refused_clearly()
+ {
+ var folder = fx.NewFolder();
+ var mdl = SourceModelBuilder.Write(folder, "box", "", ["brick", "glass"], SourceModelBuilder.TileAndTriangle());
+ File.Delete(Path.ChangeExtension(mdl, ".vvd"));
+ var missing = Assert.Throws(() => ModelConverter.Read(mdl, "mdl", null, Ct));
+ Assert.Contains("box.vvd", missing.Message);
+
+ mdl = SourceModelBuilder.Write(folder, "box2", "", ["brick", "glass"], SourceModelBuilder.TileAndTriangle(), vvdChecksum: 42);
+ var other = Assert.Throws(() => ModelConverter.Read(mdl, "mdl", null, Ct));
+ Assert.Contains("checksum", other.Message);
+
+ var notModel = Path.Combine(folder, "quake.mdl");
+ File.WriteAllBytes(notModel, "IDPO"u8.ToArray());
+ File.WriteAllBytes(Path.Combine(folder, "quake.vvd"), []);
+ File.WriteAllBytes(Path.Combine(folder, "quake.dx90.vtx"), []);
+ Assert.Throws(() => ModelConverter.Read(notModel, "mdl", null, Ct));
+ }
+
+ [Fact]
+ public void Material_files_are_read_as_keyvalues()
+ {
+ var tokens = Vmt.Tokens("""
+ "UnlitGeneric" // shader
+ {
+ $basetexture "a/b c"
+ "$alphatest" 1
+ }
+ """);
+ Assert.Equal(["UnlitGeneric", "{", "$basetexture", "a/b c", "$alphatest", "1", "}"], tokens);
+ }
+
+ private sealed class Vector3Comparer : IEqualityComparer
+ {
+ public bool Equals(Vector3 a, Vector3 b) => Vector3.Distance(a, b) < 0.01f;
+ public int GetHashCode(Vector3 v) => 0;
+ }
+}
diff --git a/tests/Filee.Engines.Tests/VpkTests.cs b/tests/Filee.Engines.Tests/VpkTests.cs
new file mode 100644
index 000000000..962f68a13
--- /dev/null
+++ b/tests/Filee.Engines.Tests/VpkTests.cs
@@ -0,0 +1,232 @@
+// Valve paks (.vpk, #1526): the directory file lists the files; their data follows the tree in it (archive 0x7FFF) or is
+// in "_nnn.vpk" next to it, after a few preload bytes in the tree. The test paks are built here byte by byte;
+// a PS Vita .vpk is a ZIP and goes to 7-Zip.
+
+using System.Buffers.Binary;
+using System.IO.Compression;
+using System.Text;
+using Filee.Core.Conversion;
+using Filee.Core.Formats;
+using Filee.Core.Presets;
+using Filee.Engines.Archives;
+using static Filee.Engines.Tests.ArchiveBuilders;
+
+namespace Filee.Engines.Tests;
+
+public class VpkTests(EngineFixture fx) : IClassFixture
+{
+ private static CancellationToken Ct => TestContext.Current.CancellationToken;
+
+ private void SkipWithout7Zip()
+ {
+ var engine = fx.Converters.Single(c => c.Id == "archive");
+ Assert.SkipUnless(fx.Catalog.StatusOf(engine).IsAvailable, "7-Zip not found (pwsh build/fetch-engines.ps1 -Only 7zip)");
+ }
+
+ private async Task ConvertOkAsync(string source, string presetId)
+ {
+ var job = await fx.ConvertAsync([source], BuiltInData.CreatePresets().Single(p => p.Id == presetId));
+ Assert.True(job.State == JobState.Completed, string.Join("; ", job.Files.Select(f => $"{f.ErrorKey} {f.ErrorDetail}")));
+ return Assert.Single(job.Outputs);
+ }
+
+ /// A file in the pak: its folder, name and extension, content, preload length and where its data goes.
+ private sealed record PakFile(string Folder, string Name, string Extension, byte[] Content, int Preload, int Archive);
+
+ private const int InDirectory = 0x7FFF;
+
+ ///
+ /// Writes "_dir.vpk" (version ) and its "_000.vpk" part into
+ /// ; returns the directory file.
+ ///
+ private static string BuildPak(string folder, string name, int version, params PakFile[] files)
+ {
+ var tree = new MemoryStream();
+ var inDirectory = new MemoryStream();
+ var part = new MemoryStream();
+ void Text(string s) => tree.Write(Encoding.ASCII.GetBytes(s + "\0"));
+ var fields = new byte[18];
+ foreach (var byExtension in files.GroupBy(f => f.Extension))
+ {
+ Text(byExtension.Key);
+ foreach (var byFolder in byExtension.GroupBy(f => f.Folder))
+ {
+ Text(byFolder.Key);
+ foreach (var file in byFolder)
+ {
+ Text(file.Name);
+ var data = file.Content[file.Preload..];
+ var target = file.Archive == InDirectory ? inDirectory : part;
+ BinaryPrimitives.WriteUInt32LittleEndian(fields, Crc32.Compute(file.Content));
+ BinaryPrimitives.WriteUInt16LittleEndian(fields.AsSpan(4), (ushort)file.Preload);
+ BinaryPrimitives.WriteUInt16LittleEndian(fields.AsSpan(6), (ushort)file.Archive);
+ BinaryPrimitives.WriteUInt32LittleEndian(fields.AsSpan(8), (uint)target.Length);
+ BinaryPrimitives.WriteUInt32LittleEndian(fields.AsSpan(12), (uint)data.Length);
+ BinaryPrimitives.WriteUInt16LittleEndian(fields.AsSpan(16), 0xFFFF);
+ tree.Write(fields);
+ tree.Write(file.Content.AsSpan(0, file.Preload));
+ target.Write(data);
+ }
+ Text("");
+ }
+ Text("");
+ }
+ Text("");
+
+ var dir = new MemoryStream();
+ var header = new byte[version == 2 ? 28 : 12];
+ BinaryPrimitives.WriteUInt32LittleEndian(header, 0x55AA1234);
+ BinaryPrimitives.WriteUInt32LittleEndian(header.AsSpan(4), (uint)version);
+ BinaryPrimitives.WriteUInt32LittleEndian(header.AsSpan(8), (uint)tree.Length);
+ if (version == 2)
+ BinaryPrimitives.WriteUInt32LittleEndian(header.AsSpan(12), (uint)inDirectory.Length);
+ dir.Write(header);
+ tree.WriteTo(dir);
+ inDirectory.WriteTo(dir);
+ var path = Path.Combine(folder, name + "_dir.vpk");
+ File.WriteAllBytes(path, dir.ToArray());
+ if (part.Length > 0)
+ File.WriteAllBytes(Path.Combine(folder, name + "_000.vpk"), part.ToArray());
+ return path;
+ }
+
+ private static PakFile[] SampleFiles =>
+ [
+ new("materials/brick", "brickwall001", "vtf", Noise(5000, 1), 0, 0),
+ new("materials/brick", "brickwall001", "vmt", Encoding.ASCII.GetBytes("\"LightmappedGeneric\" { \"$basetexture\" \"brick/brickwall001\" }"), 16, InDirectory),
+ new("scripts", "game_sounds", "txt", Words(700), 0, InDirectory),
+ new(" ", "readme", " ", Encoding.ASCII.GetBytes("top level, no extension"), 0, 0),
+ ];
+
+ [Fact]
+ public void Vpk_is_recognised_and_unpacks_in_one_step()
+ {
+ Assert.Equal("vpk", FormatRegistry.Detect("pak01_dir.VPK")?.Id);
+ var planner = fx.Catalog.CreatePlanner(["archive"]);
+ Assert.Single(planner.Plan("vpk", FormatRegistry.Folder)!.Steps);
+ Assert.Single(planner.Plan("vpk", "zip")!.Steps);
+ Assert.False(FormatRegistry.Get("vpk")!.Writable);
+ }
+
+ [Theory]
+ [InlineData(1)]
+ [InlineData(2)]
+ public async Task A_pak_unpacks_with_its_preload_bytes_and_its_parts(int version)
+ {
+ SkipWithout7Zip();
+ var dir = BuildPak(fx.NewFolder(), "pak01", version, SampleFiles);
+
+ var folder = await ConvertOkAsync(dir, "extract");
+
+ foreach (var file in SampleFiles)
+ {
+ var name = (file.Folder.Trim() is "" ? "" : file.Folder + "/") + file.Name + (file.Extension.Trim() is "" ? "" : "." + file.Extension);
+ Assert.Equal(file.Content, File.ReadAllBytes(Path.Combine(folder, name)));
+ }
+ }
+
+ [Fact]
+ public async Task A_pak_becomes_a_zip()
+ {
+ SkipWithout7Zip();
+ var dir = BuildPak(fx.NewFolder(), "hl2_textures", 2, SampleFiles);
+
+ var zip = await ConvertOkAsync(dir, "to-zip");
+
+ using var archive = ZipFile.OpenRead(zip);
+ Assert.Equal(["materials/brick/brickwall001.vmt", "materials/brick/brickwall001.vtf", "readme", "scripts/game_sounds.txt"],
+ archive.Entries.Where(e => !e.FullName.EndsWith('/')).Select(e => e.FullName).Order(StringComparer.Ordinal));
+ }
+
+ [Fact]
+ public void A_missing_part_a_data_part_and_a_bad_checksum_are_refused_clearly()
+ {
+ var folder = fx.NewFolder();
+ var dir = BuildPak(folder, "pak01", 2, SampleFiles);
+ var target = Directory.CreateDirectory(Path.Combine(folder, "out")).FullName;
+
+ var part = Path.Combine(folder, "pak01_000.vpk");
+ var error = Assert.Throws(() => Vpk.Extract(part, target, Ct));
+ Assert.Contains("pak01_dir.vpk", error.Message);
+
+ File.Move(part, part + ".bak");
+ var missing = Assert.Throws(() => Vpk.Extract(dir, Directory.CreateDirectory(Path.Combine(folder, "out2")).FullName, Ct));
+ Assert.Contains("pak01_000.vpk", missing.Message);
+ File.Move(part + ".bak", part);
+
+ // Change one byte of the texture's data in the part.
+ var bytes = File.ReadAllBytes(part);
+ bytes[100] ^= 0xFF;
+ File.WriteAllBytes(part, bytes);
+ Assert.Throws(() => Vpk.Extract(dir, Directory.CreateDirectory(Path.Combine(folder, "out3")).FullName, Ct));
+
+ File.WriteAllBytes(dir, File.ReadAllBytes(dir)[..20]);
+ Assert.Throws(() => Vpk.Extract(dir, Directory.CreateDirectory(Path.Combine(folder, "out4")).FullName, Ct));
+ }
+
+ [Theory]
+ [InlineData("../..", "evil")]
+ [InlineData("/etc", "evil")]
+ [InlineData("C:/Windows", "evil")]
+ public void A_name_leaving_the_output_folder_is_refused(string folderName, string name)
+ {
+ var folder = fx.NewFolder();
+ var dir = BuildPak(folder, "evil", 1,
+ new PakFile("lua", "fine", "lua", "x = 1"u8.ToArray(), 0, InDirectory),
+ new PakFile(folderName, name, "lua", "boom"u8.ToArray(), 0, InDirectory));
+ var target = Directory.CreateDirectory(Path.Combine(folder, "a", "b", "out")).FullName;
+
+ var error = Assert.Throws(() => Vpk.Extract(dir, target, Ct));
+
+ Assert.Contains("outside the target folder", error.Message);
+ Assert.Empty(Directory.EnumerateFileSystemEntries(target)); // nothing written, not even the safe file
+ }
+
+ [Fact]
+ public void A_tree_pointing_outside_itself_is_refused()
+ {
+ var folder = fx.NewFolder();
+ var dir = BuildPak(folder, "pak01", 1, new PakFile("lua", "a", "lua", Words(100), 10, InDirectory));
+ var good = File.ReadAllBytes(dir);
+ var fieldsAt = good.AsSpan().IndexOf("\0a\0"u8) + 3; // after the name "a", behind "lua\0lua\0"
+
+ // Preload bytes running past the end of the tree.
+ var preload = (byte[])good.Clone();
+ BinaryPrimitives.WriteUInt16LittleEndian(preload.AsSpan(fieldsAt + 4), 60000);
+ File.WriteAllBytes(dir, preload);
+ Assert.Throws(() => Vpk.Extract(dir, Directory.CreateDirectory(Path.Combine(folder, "a")).FullName, Ct));
+
+ // A data part beyond "_999".
+ var index = (byte[])good.Clone();
+ BinaryPrimitives.WriteUInt16LittleEndian(index.AsSpan(fieldsAt + 6), 4000);
+ File.WriteAllBytes(dir, index);
+ Assert.Throws(() => Vpk.Extract(dir, Directory.CreateDirectory(Path.Combine(folder, "b")).FullName, Ct));
+
+ // A tree larger than Filee reads.
+ var tree = (byte[])good.Clone();
+ BinaryPrimitives.WriteUInt32LittleEndian(tree.AsSpan(8), uint.MaxValue);
+ File.WriteAllBytes(dir, tree);
+ Assert.Throws(() => Vpk.Extract(dir, Directory.CreateDirectory(Path.Combine(folder, "c")).FullName, Ct));
+ }
+
+ [Fact]
+ public async Task A_ps_vita_package_is_a_zip()
+ {
+ SkipWithout7Zip();
+ var folder = fx.NewFolder();
+ var vpk = Path.Combine(folder, "homebrew.vpk");
+ // A real package has eboot.bin beside sce_sys, so the folder is not flattened.
+ using (var zip = ZipFile.Open(vpk, ZipArchiveMode.Create))
+ {
+ using (var entry = zip.CreateEntry("eboot.bin").Open())
+ entry.Write(Noise(32, 8));
+ using (var entry = zip.CreateEntry("sce_sys/param.sfo").Open())
+ entry.Write(Noise(64, 9));
+ }
+
+ var unpacked = await ConvertOkAsync(vpk, "extract");
+
+ Assert.Equal(Noise(32, 8), File.ReadAllBytes(Path.Combine(unpacked, "eboot.bin")));
+ Assert.Equal(Noise(64, 9), File.ReadAllBytes(Path.Combine(unpacked, "sce_sys", "param.sfo")));
+ }
+}