feat: int8-static export builds and gates the browser-size composition - #224
Merged
Merged
Conversation
build_int8_static_zip in imf.export: dynamic-int8 encoder + static-int8
decoder in one zip, calibrated on the model's own fp32 decode across
framings (prefill, single steps, 8-token windows); metadata precision
rewritten to int8 (PRECISIONS is closed — the recipe is the zip name,
the gate the value). Fixture specs cover the release sequence end to
end: build, parity gate, write_parity, strict validation, both quant
recipes present, head MatMul kept fp32.
modal_export::export_int8_static becomes thin orchestration over the
helper plus the rebuild_int8_head32 gate stack (reference decode,
parity, margin analysis, confident-flip budget), landing
{mid}-int8static.zip on the volume.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Closes the export half of TODO.impl/11's positive branch: the browser-size static-int8 composition, built and gated on the release machinery.
imf.export.build_int8_static_zip: dynamic-int8 encoder + static-int8 decoder in one zip. Calibration walks the model's own fp32 decode over eval inputs across framings (prefill, single-token steps, 8-token windows) — the shapes whose activation scales must hold (the framing axis, Paper C §5.1). Metadata precision is rewritten toint8:PRECISIONSis a closed set, so the static recipe is carried by the zip name and the gate (int8's 2pp cer_delta limit) by the value.modal_export::export_int8_staticbecomes thin orchestration over the helper plus therebuild_int8_head32gate stack: reference decode (resume-safe), CER parity written into the zip, margin analysis, confident-flip ≤1% budget. Lands{mid}-int8static.zipon the models volume; publication stays a separate version decision.Specs
Fixture end-to-end over the tiny T5 (no mocks): build → parity gate (500+ samples) →
write_parity→ strict validation; both quant recipes present in the graphs (MatMulInteger encoder, QLinearMatMul decoder); head MatMul kept fp32 per the E1 rule.Context
Measured: composition B 4.6045 full-set vs dynamic 4.5701 (paired CI not separated), +8% CPU decode, 491 MiB. This branch only adds the gated build path; the
staticentrypoint runs it forara-diac-small-21.