Repository navigation
Conversation
GMA versions 1 to 3 unpack to a folder or any archive. Every name is checked before anything is written, the entries are capped and each size must fit in what is left of the addon; every file's CRC is checked (0 from gmad -nocrc is not). An addon.json with the title, description, type and tags is written beside the files, as gmad reads it to pack the folder again. GMA is read-only here and in the Games field; .gma files learn the archives donut (schema 95). Part of KnifeLemon#1526.
The directory file of a Valve pak (versions 1 and 2) unpacks to a folder or any archive: the files' preload bytes and their data, in the directory file or in the _000.vpk ... parts next to it, every CRC checked. A data part dropped on its own is refused with the name of its directory file; a missing part names the part. A PS Vita package, a ZIP under the same extension, goes to 7-Zip. Every name is checked before anything is written; the tree is capped at 256 MB and the entries at the archive limit, preload bytes must lie inside the tree and parts are _000 to _999. VPK is read-only and in the Games field; .vpk files learn the archives donut (schema 95). Part of KnifeLemon#1526.
The custom textures, models and sounds packed into a Source engine map are a ZIP in its pakfile lump (lump 40). Filee finds it, Left 4 Dead 2's lump order included, copies it out and 7-Zip unpacks it, so a map extracts or converts to any archive. A map without packed files, a compressed pakfile lump and a Quake or GoldSrc map are refused with a message saying so. BSP is in the Games field; .bsp files learn the archives donut (schema 95). Part of KnifeLemon#1526.
A folder, a ZIP of one or the files dropped onto Compress into one archive pack into a GMA as gmad does: addon.json for the title, type, tags and ignores, only the files on gmad's whitelist, lower-case names, a CRC per file and over the whole addon. A folder with nothing Garry's Mod loads is refused. GMA becomes writable; the GMA preset joins the game packs in the palette after MRPACK, not a donut (schema 96). Part of KnifeLemon#1526.
A Source engine model (versions 44 to 49, with the .vvd and .dx90.vtx beside it, all three of the same build) becomes a 3D model in its bind pose with its default body groups, first skin and LOD 0, turned to Filee's Y up, millimetres, counter-clockwise and bottom-left UVs, so it converts to OBJ, GLB, STL and the other 3D formats and to a picture. Its textures are found as Garry's Mod finds them: the .vmt in the materials folder beside the models folder, its $basetexture decoded from VTF. MDL is in the Games field; .mdl files learn the models donut (schema 97). Part of KnifeLemon#1526.
… skeleton An MDL becomes a reference SMD as Crowbar decompiles it: its bones with their bind pose, its triangles counter-clockwise with up to three bone weights per vertex, UVs from the bottom left, in inches, Z up. Any other 3D model becomes a static prop with one root bone, ready for studiomdl; a reference SMD's triangles become a model again, so SMD converts to GLB, OBJ, STL and the rest. SMD is in the Games field; .smd files learn the models donut and the SMD preset joins the 3D models in the palette (schema 97). Part of KnifeLemon#1526.
The "QC + SMD" preset turns a .mdl with its .vvd, .vtx, .phy and .ani into the folder studiomdl builds it from, as Crowbar decompiles: the QC (body groups, LODs, materials and skins, attachments, hitboxes, pose parameters, sequences with their events, collision model or joints, key values), a reference SMD per body group model, an SMD per LOD that differs, the physics SMD and an animation SMD per animation. Animations are decoded from the .mdl or its .ani (raw and run-length encoded rotations and positions, sections, root motion); the .phy's convex pieces are walked through their ledge trees. Every count is capped and every offset checked before it is used; a damaged file is refused and the folder removed. The preset is in the palette after "SMD", not on a donut (schema 98). Part of KnifeLemon#1526.
ad970b5 to
9351b5c
Compare
KnifeLemon
left a comment
There was a problem hiding this comment.
Thanks, @ltsammy, this is a big piece of work. The shared SourceBytes reader, the frame, track and ledge-tree caps and the non-recursive tree walk are what we want. I read the new commit on top of #1609. CI's one red job (WebDropJobTests on Linux x64) is a Core test for code this PR doesn't touch; I re-ran it.
Every count has its own cap, but as in #1609, many records can point at the same bytes, so the caps multiply. A .mdl of a few KB can still ask for terabytes of output or all the memory. A decompile needs one budget for the whole model. Could you add one, counted as things are read or written, that refuses with ConversionRefusedException("error.document_too_large", "…") (as OdsReader line 345 and Ebooks/Book.cs line 98 do) when it's exceeded? These are the places:
1. Animations (the biggest one)
src/Filee.Engines/Models3D/SourceAnimation.cs, Decode (lines 51–89) keeps every frame of every bone in a List<Pose[]> before anything is written.
- An animation needs no data at all to be long: with block 0 and index 0,
Locatereturns −1 and every frame is the bind pose. - So one 100-byte animation record with 100,000 frames on a 1,024-bone skeleton is about 2.4 GB of poses in memory and about 6 GB of SMD text.
ReadAnimations(SourceModelDetails.cs) allows 16,384 such records.
Please cap the bone-frames (frames × bones) across all animations; tens of millions is far above any real model. If you can, also write each frame as it is decoded instead of holding them all.
2. Triangles over every body group model and LOD
The point from #1609 applies here across the whole decompile. SourceModelDetails.ReadBodyParts (lines 164–192) calls SourceModel.ReadLodMeshes (SourceModel.cs, from line 240) for up to 512 parts × 512 models × 8 LODs. Strip groups and strips inside each LOD can share one index table. One triangle budget for all of them covers it.
3. Physics
src/Filee.Engines/Models3D/SourcePhysics.cs, ReadSolid (lines 69–96): every leaf of the ledge tree can point at the same ledge (node + I32(node + 4)). So each of the 1,024 solids can hold about 32,000 copies of a piece with up to 32,767 triangles. A cap on the triangles (or points) of the whole .phy covers it.
4. QC entries
These end up in memory and in the QC's StringBuilder:
ReadSequences(SourceModelDetails.cs:267): 16,384 sequences × 4,096 events.ReadHitboxes(SourceModelDetails.cs:212): 4,096 sets × 16,384 hitboxes.
Each is up to 67 million entries, and their tables can overlap too. Caps on the totals (all events, all hitboxes) would close this.
5. Small: Windows device names
SourceDecompiler.SafeName (SourceDecompiler.cs:472) keeps a name such as con, nul, aux or com1. Windows treats those as devices, and on Windows 10 even with an extension, so an animation or body model with that name can't be written as nul.smd and the like. Prefixing them with _, as ArchiveSafety.SanitizeSegment does, avoids it.
6. Reference credits in comments
As in #1609, please keep comments to the format and to compatibility ("as studiomdl reads it", "the layout Crowbar writes" is fine). Drop where the layout was learned, for example SourcePhysics.cs:3 ("and SourceIO's reader").
Main has moved
#1585 took schema step 95 on main, so I merged main into #1591 and #1608 and moved their steps to 96 and 97. The VTF preset (#1611) gets 98. SMD and "QC + SMD" take the next two free numbers when #1609 and this PR land. If the renumbering gets in your way, push your fixes as they are and we'll do it before merging.
What does this change?
Follow-up to #1609 for #1526: a Source engine model can now be decompiled into the folder studiomdl builds it from, the way Crowbar does. The "QC + SMD" preset turns a
.mdl(with its.vvd,.vtx,.phyand.ani) into a folder named after the model:<model>.qc$modelname, model flags,$surfaceprop,$illumposition,$bbox/$cbox,$bodygroups,$lod/$shadowlod,$cdmaterials,$texturegroup,$attachments,$hboxset/$hboxes,$includemodels,$poseparameters, a$sequenceper sequence (animations, blends, activity, fps, options, motion flags, fades, events),$collisionmodelor$collisionjoints(with$jointconstrain),$keyvalues<body model>.smdblank)<body model>_lod<n>.smdremovemodel<model>_physics.smd.phy<model>_anims/<animation>.smdHow it reads them
LODs use the LOD 0 vertices with each LOD's own triangles from the
.vtx. This was checked against a six-LOD Garry's Mod vehicle.Animations come from the
.mdl, or from the.anibeside it when they're in animation blocks, section by section. Each animated bone can have:Track values are scaled by the bone's scales and added to its bind pose (except in delta animations). The root motion studiomdl took out (movements) is added back.
Physics walks the
.phy's IVP ledge trees and turns metres in IVP's axes into inches. Solids named after bones (a ragdoll) are moved from bone space into model space.Not written: flexes (
.vta), eyeballs, IK rules and chains, bone controllers and procedural bones. A model with them still compiles from the QC, just without them. Compiling back to.mdlstill needs Valve's studiomdl.Hostile files
SourceBytes)..physolids (1,024), ledge-tree nodes (65,536) and points per piece (65,536).Settings
.mdlcan take it. The name is untranslated like "SMD" and "GMA", so there are no new UI strings.Tests
SourceDecompileTests): the test model builder now also writes an attachment, a second LOD, a three-frame animation (Quaternion64, Vector48 and an RLE rotation track) with a sequence and an event, and a.phywith one convex piece. The tests cover:SourceFormatTests): the v97 → v98 migration (preset after "SMD", not on a donut).Type of change
How was it tested?
dotnet format --verify-no-changesdotnet testChecklist
src/Filee.App/Assets/i18n/*.jsonfiles (none added)THIRD-PARTY-NOTICES.md(none added)