A simple Model Compression Lab.
Train one large baseline Multilayer Perceptron (MLP) on a randomized CSV dataset, then shrink it with four techniques and compare size, accuracy, and latency:
| Technique | What it does | File |
|---|---|---|
| Dynamic range quantization | int8 weights; activations quantized at inference from their observed range (no calibration data) | quantization.py |
| Weight pruning | Global magnitude pruning at several sparsities, with and without masked fine-tuning | pruning.py |
| Model distillation | Small student trained on a teacher's softened logits vs. hard labels only | distillation.py |
| Dimensionality reduction | PCA input compression (64 → 32/16/8/4 features) | dimensionality_reduction.py |
Stages run sequentially: generate data → train baseline → each technique. Models are written to lab/models, results to lab/metrics (CSV + JSON).
docker compose up
- Quantization: ~3.8x smaller, accuracy nearly unchanged. Dynamic quantization is simulated in float (portable); real latency gains need an int8 kernel.
- Pruning: accuracy collapses at high sparsity before fine-tuning and largely recovers after. Sparse weights only shrink the file once compressed (
gzip_size_kb) or stored in a sparse format. - Distillation: the student has ~1.7% of the teacher's parameters; distillation gives a modest edge over hard labels only (small and seed-dependent).
- Dimensionality reduction: 16 components keep ~97.6% variance with no accuracy loss; 8 or fewer drop sharply. Note the parameter savings are small since only the first layer shrinks.
Generated data, models, and metrics are included for study; they are recreated on each run.
The above frequently show up in the GCP PMLE but there aren't that many hands-on labs for these in skills.google.