ZSharp

Sharpness-Aware Minimization with Z-Score Gradient Filtering: +5.26% accuracy over SGD, Apple Silicon optimized, fully reproducible

Quickstart

Clone the repo and run the demo:

git clone https://github.com/bangyen/zsharp.git
cd zsharp
pip install -e .
pytest   # optional: run tests
python -m scripts.train --config configs/zsharp_baseline.yaml

Or open in Colab: Colab Notebook.

Results

Scenario / Dataset	Baseline	This Project	Δ Improvement
CIFAR-10 ResNet-18	74.89%	80.15%*	+5.26%

*Benchmark results from full training runs. Local results may vary based on configuration.

Note: this benchmark predates the alignment of the implementation to the paper (see docs/algorithm.md). It was produced with 70th-percentile filtering, an SGD base optimizer, and gradient clipping, and has not been regenerated under the current defaults.

Features

• Z-Score Gradient Filtering — Layer-wise Z-score normalization with a global 95th percentile threshold (configurable), matching the paper's $Q_p = 0.95$.
• Apple Silicon Optimization — Up to 4.39x speedup using MPS (Metal Performance Shaders) for faster training on Mac.
• Paper Architectures — CIFAR-style ResNet-56/110, VGG-16BN, and the paper's compact ViTs, on CIFAR-10/100 and Tiny-ImageNet.
• Comprehensive Testing — 95%+ test coverage with 85 unit tests ensuring reliability and reproducibility.

Repo Structure

zsharp/
├── zsharp_demo.ipynb  # Colab notebook demo
├── scripts/           # Training and experiment scripts
├── tests/             # Unit/integration tests (85 tests)
├── docs/              # Documentation and training curves
├── configs/           # Configuration files
├── results/           # Experimental results
└── zsharp/               # Core implementation

Validation

• ✅ 95%+ test coverage (pytest)
• ✅ Reproducible seeds for experiments
• ✅ Benchmark scripts included

References

• Sharpness-Aware Minimization with Z-Score Gradient Filtering — Original research paper by Juyoung Yun. The optimizer and default hyperparameters follow this paper: $Q_p = 0.95$, $\rho = 0.05$, AdamW base optimizer (lr 1e-3, weight decay 5e-5), and an LR step decay of 0.75 every 10 epochs.
• Sharpness-Aware Minimization — Foundation SAM algorithm research.

License

This project is licensed under the MIT License.