Skip to content
IntechgentPublic

About

Deep learning image classifier for skin-lesion diagnosis in PyTorch, with a focus on handling severe class imbalance in medical data.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

22 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SkinSight

A deep-learning image classifier for skin-lesion diagnosis, built in PyTorch on the DermaMNIST (HAM10000) dataset. The project trains models to classify dermatoscopic images into 7 lesion types, with a focus on the central challenge of the dataset: severe class imbalance, where the naive "high accuracy" is dangerously misleading.

Per-class recall across all four models

The problem

DermaMNIST contains 7 classes of skin lesion, but they are wildly imbalanced: melanocytic nevi make up 67% of the data, while dermatofibroma is just 1.1%. This creates a trap: a model that ignores the images and always predicts the majority class scores 67% accuracy while learning nothing useful.

This matters clinically, not just statistically. The rare classes include melanoma, the most dangerous lesion. A model that plays the odds by ignoring rare classes will systematically miss the cases that matter most. So the real goal is not headline accuracy, but per-class recall, especially on the dangerous minority classes.

Approach

I ran a controlled 2x2 experiment, varying two factors independently:

  • Architecture: a small from-scratch fully-connected network (TinyNet) vs a pretrained ResNet-18 (transfer learning, frozen body + fine-tuned head).
  • Loss: standard cross-entropy vs inverse-frequency weighted cross-entropy (making rare-class errors cost more).

This isolates the effect of each factor and tests whether better features (transfer learning) and imbalance handling (weighting) are the same thing or separate, complementary tools.

Results

Per-class recall (%), validation set:

Class TinyNet TinyNet + weighted ResNet ResNet + weighted
actinic keratoses 15.2 42.4 66.7 69.7
basal cell carcinoma 55.8 57.7 17.3 57.7
benign keratosis 26.4 41.8 26.4 25.5
dermatofibroma 0.0 25.0 33.3 41.7
melanoma 18.9 37.8 42.3 55.9
nevi (majority) 94.6 50.4 95.1 57.1
vascular 21.4 71.4 21.4 78.6
overall accuracy 72.0 48.2 75.0 54.0

Key findings:

  1. Accuracy is misleading here. The plain TinyNet's 72% comes almost entirely from the majority class (94.6% recall on nevi) while it misses 81% of melanomas. High accuracy, poor screening tool.

  2. Feature quality and imbalance handling are separate problems. Transfer learning raised overall accuracy and melanoma recall, but the plain ResNet still leaned on the majority class (95.1% nevi). Better features alone did not solve imbalance.

  3. The two techniques stack. ResNet + weighted loss catches more melanomas than either technique alone, while staying balanced across rare classes. Its overall accuracy is lower than plain ResNet, but it is the more useful model for screening, where a missed melanoma is far costlier than a false alarm.

Evaluation

Beyond recall, this model was evaluated with a confusion matrix and full precision/recall/F1 analysis:

Normalised confusion matrix

The model trades precision for recall on dangerous classes: it catches 56% of melanomas but with many false alarms (29% precision). The confusion matrix shows its errors are mostly "safe" ones, confusing melanoma with other malignant classes rather than benign ones, and over-flagging benign moles as suspicious. For a screening context, erring toward caution is the defensible bias.

Misclassified validation images

The failure gallery bears this out. Almost every mistake is a benign nevus called something worse, and plenty of those moles do look dark and irregular. The source images are 28x28 upscaled to 224, so the fine texture that separates lesions is blurred out before the model ever sees it.

Macro-average F1 is 0.376 against a weighted-average of 0.596. That spread is the model still doing better on common classes than on rare ones.

Phase 2: native resolution and partial fine-tuning

The evaluation flagged two things holding the model back. The images were upscaled from 28x28, so the texture that separates lesion types was gone before the model saw them. And the ResNet body was frozen solid, so its features never adjusted from ImageNet to skin. I tried both fixes.

Native resolution. MedMNIST also publishes the images resized straight from the original HAM10000 photos to 224px, instead of the 28px thumbnails stretched up. The model was already resizing everything to 224px anyway, so the compute is identical; the only difference is whether the pixels are real or interpolated.

Partial fine-tuning. A full unfreeze is 11.2M parameters and too slow on CPU, so I unfroze just the last residual block (layer4) and the head, about 8.4M parameters. That lets the deepest features adapt while the early edge and texture filters stay as ImageNet left them. One catch: the still-frozen blocks have to run in eval() mode, because requires_grad = False stops their weights updating but not their BatchNorm running stats. The learning rate also drops 10x to 1e-4 so the pretrained weights survive the first few steps.

Stage Overall accuracy Melanoma recall Melanoma precision Macro F1
ResNet + weighted (upscaled, frozen) 54.0 55.9 29.1 0.376
+ native 224px resolution (frozen) 68.6 64.9 n/a n/a
+ partial fine-tuning (layer4 unfrozen) 77.0 73.9 43.2 0.680

(The frozen-native run only recorded per-class recall. Precision and macro F1 are from the fine-tuned model's classification report.)

Resolution alone added 14.6 points of accuracy at no extra cost. Fine-tuning added another 8.4 on top. Melanoma is the clearest case: recall went from 56% to 74% and precision from 29% to 43% together, so the model catches more real melanomas while raising fewer false alarms per flag. Usually one of those comes at the expense of the other. The gap between macro and weighted F1, which had been the clearest sign of a model coasting on the majority class, closed from 22 points to 10.6. Every class now sits above 58% recall; in the upscaled model the weakest was 25%.

Fine-tuning drove train loss down hard, 1.11 to 0.20, which on its own would read as overfitting. But validation recall went up on all seven classes, not just the easy ones, so this was the model learning better features rather than memorising the training set.

What I would improve

  • Unfreeze the whole ResNet body, not just layer4. The remaining frozen layers would probably give a little more, but a full fine-tune needs a GPU to be practical.
  • Augment or oversample the rare classes to pull melanoma precision up further without giving back recall.
  • Tune the class-weight strength. It over-flags right now; there is likely a setting that trades a little recall for much less noise.
  • Build a confusion matrix and failure gallery for the fine-tuned model. The first evaluation showed melanoma being confused with other malignant classes rather than benign ones, and it is worth checking whether that still holds.

Repository structure

SkinSight/
├── notebooks/          # step-by-step build, from PyTorch fundamentals to evaluation
│   ├── 00_tensors.ipynb ... 02_training_loop.ipynb   # fundamentals (built from scratch)
│   ├── 03_data.ipynb                                 # data loading + imbalance analysis
│   ├── 04_baseline_and_imbalance.ipynb               # weighted-loss experiment
│   ├── 05_transfer_learning.ipynb                    # ResNet transfer learning
│   ├── 06_evaluation.ipynb                           # confusion matrix, F1, failure gallery
│   └── 07_higher_resolution.ipynb                    # native resolution + partial fine-tuning
├── src/                # reusable, importable pipeline
│   ├── model.py        # TinyNet + ResNet builder
│   ├── data.py         # transforms, loaders, class weights
│   └── train.py        # train / evaluate / seed functions
└── requirements.txt

Running it

pip install -r requirements.txt

The src/ module runs the whole pipeline in a few lines. The frozen ResNet + weighted-loss baseline from the 2x2 experiment:

from src.model import build_resnet
from src.data import get_loaders, compute_class_weights
from src.train import set_seed, train, evaluate

set_seed(42)
train_loader, val_loader, train_data = get_loaders()
weights = compute_class_weights(train_data)

model = build_resnet(num_classes=7, freeze_body=True)
model = train(model, train_loader, class_weights=weights, epochs=3)
accuracy, per_class_recall = evaluate(model, val_loader)

The Phase 2 model (native resolution, layer4 fine-tuned) is the same three calls with different arguments:

set_seed(42)
train_loader, val_loader, train_data = get_loaders(native=True)
weights = compute_class_weights(train_data)

model = build_resnet(num_classes=7, freeze_body=True, unfreeze_layer4=True)
model = train(model, train_loader, class_weights=weights, epochs=3, lr=1e-4, freeze_bn=True)
accuracy, per_class_recall = evaluate(model, val_loader)

Or work through the notebooks in order to see the full build, from tensor basics to the final evaluation.

Tech stack

PyTorch, torchvision, scikit-learn, medmnist, matplotlib, NumPy.

Note

This is an educational project on a public benchmark dataset. It is not a medical device and must not be used for actual diagnosis.

License

This project is licensed under the MIT License - see the LICENSE file for details.

The DermaMNIST dataset is distributed under CC BY-NC 4.0 (non-commercial). The underlying HAM10000 images are credited to their original authors.

About

Deep learning image classifier for skin-lesion diagnosis in PyTorch, with a focus on handling severe class imbalance in medical data.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages