MalwareScope — Static PE Malware Detection

A machine learning system that classifies Windows PE executable files as malware or goodware using static analysis features. Built with LightGBM, Flask, and deployed with a full CI/CD pipeline.

---

Table of Contents

• Overview
• Project Structure
• Dataset
• Quickstart
• Training the Model
• Running the Web App
• Web App Features
• Model Results
• CI/CD Pipeline
• Automated Tests
• Deployment
• File Descriptions
• AI Tooling

---

Overview

MalwareScope performs static malware detection — classifying a PE (Portable Executable) .exe file as malware or goodware based purely on its static features, without ever executing it. This is useful because it allows prediction of potentially harmful software before it runs.

The project covers the full ML lifecycle:

• Exploratory data analysis and preprocessing
• Training and comparing 7 ML models with 10-fold cross-validation
• Selecting and packaging the best model for production
• Serving predictions via a Flask web application
• Automated testing and CI/CD deployment via GitHub Actions

---

Project Structure

malware-ml-project/
│
├── data/
│   └── test_set.csv              # 20% hold-out test set (saved after split)
│
├── model/
│   |└── model.pkl   # Trained LightGBM pipeline (joblib) 
    └── le.pkl                    #LabelEncoder            
├── templates/
│   ├── index.html                # Home page — manual form + CSV upload
│   ├── result.html               # Single prediction result page
│   └── upload_result.html        # Batch prediction results + metrics
│
├── tests/
│   ├── __init__.py
│   ├── test_preprocessing.py     # Unit tests — model loading and predictions
│   ├── test_app.py               # Integration tests — Flask routes
│   └── test_smoke.py             # Post-deploy smoke test — /health endpoint
│
├── .github/
│   └── workflows/
│       └── ci-cd.yml             # GitHub Actions CI/CD pipeline
│
├── config.py                     # Global constants (SEED, paths)
├── train.py                      # Full training pipeline — run to reproduce
├── eval.py                       # Final test set evaluation
├── app.py                        # Flask web application
├── requirements.txt              # Pinned dependencies
├── deployed.md                   # Live deployment URL
├── evaluation-and-design.md      # Full CV results + design decisions
└── ai-tooling.md                 # AI tools used during development

---

Dataset

Brazilian Malware Dataset — Source

Property	Value
Format	CSV
Total samples	50,181
Features	27 input attributes (PE header fields)
Target	Label — 0 = Goodware, 1 = Malware
Class distribution	57.9% malware / 42.1% goodware
File type	Portable Executable (PE) — Windows .exe format

Download the dataset and place brazilian-malware.csv inside the data/ folder before running train.py.

The dataset is also hosted as a GitHub Release artifact and is downloaded automatically by the CI/CD pipeline — no manual step needed in GitHub Actions.

Feature Engineering

Five string columns required special handling:

Column	Action	Reason
SHA1	Dropped	Unique file hash — 43,411 unique values, causes overfitting
ImportedDlls	Dropped	10,813 unique values — too high cardinality
ImportedSymbols	Dropped	18,747 unique values — too high cardinality
FirstSeenDate	Dropped	Temporal leakage risk
Identify	LabelEncoded	241 compiler/packer IDs — useful signal

Final feature count: 23 numeric features

---

Quickstart

1. Clone the repo

git clone https://github.com/YOUR_USERNAME/malware-ml-project.git
cd malware-ml-project

2. Create and activate a virtual environment

# Create
python -m venv venv

# Activate — Mac/Linux
source venv/bin/activate

# Activate — Windows
venv\Scripts\activate

3. Install dependencies

pip install -r requirements.txt

4. Add the dataset

Place brazilian-malware.csv in the data/ folder.

5. Train the model

python train.py

6. Evaluate on the test set

python eval.py

7. Run the web app

python app.py
# Open: http://localhost:5000

---

Training the Model

train.py runs the full pipeline end-to-end:

Load data → Encode Identify → Split 80/20 → Save test_set.csv
→ 10-fold CV on all 7 models → Print results table
→ Select best model by AUC → Retrain on full training set
→ Save model/model.pkl

Expected runtime: 10–20 minutes depending on hardware.

Expected output:

Train size: (40144, 23)
Test size:  (10037, 23)

── Training Baseline Models ─────────────────────────────────
  Logistic Regression
  AUC:      0.9329 ± 0.0043
  Accuracy: 0.8778 ± 0.0059
  ...

── CV Results Summary ───────────────────────────────────────
Model                      AUC Mean   AUC Std   Acc Mean
----------------------------------------------------------
Logistic Regression          0.9329    0.0043     0.8778
Decision Tree                0.9792    0.0020     0.9800
Random Forest                0.9979    0.0007     0.9881
PyTorch MLP                  0.8910    0.0096     0.8248
XGBoost                      0.9977    0.0005     0.9846
LightGBM                     0.9980    0.0004     0.9858
CatBoost                     0.9962    0.0007     0.9808

✓ Best model: LightGBM (AUC=0.9980)
✓ Model saved to model/model.pkl

---

Running the Web App

python app.py

The app runs at http://localhost:5000. For production, use gunicorn:

gunicorn app:app

---

Web App Features

Single Sample Prediction (/)

• Pre-filled form with a real demo malware sample from the dataset
• Edit any feature value and click Run Prediction
• Returns: classification label (Malware / Goodware) + probability score + animated probability bar

Batch CSV Upload (/upload)

• Upload any .csv file containing feature columns
• Returns predictions for every row
• If the CSV contains a Label column (e.g. upload data/test_set.csv):
  • AUC score
  • Accuracy
  • Full confusion matrix (TP, TN, FP, FN)

Health Check (/health)

curl https://malware-detection-mtvw.onrender.com/health
# → {"status": "ok"}

---

Model Results

Cross-Validation (10-Fold Stratified CV)

Model	AUC Mean	AUC Std	Acc Mean	Acc Std
Logistic Regression	0.9329	±0.0043	0.8778	±0.0059
Decision Tree	0.9792	±0.0020	0.9800	±0.0019
Random Forest	0.9979	±0.0007	0.9881	±0.0014
PyTorch MLP	0.8910	±0.0096	0.8248	±0.0124
XGBoost	0.9977	±0.0005	0.9846	±0.0017
LightGBM ★	0.9980	±0.0004	0.9858	±0.0012
CatBoost	0.9962	±0.0007	0.9808	±0.0016

Final Hold-Out Test Set (20% — 10,037 samples)

Metric	Value
AUC	0.9976
Accuracy	98.65%
Precision (weighted)	0.99
Recall (weighted)	0.99
F1-Score (weighted)	0.99

Confusion Matrix

                   Predicted Goodware    Predicted Malware
Actual Goodware         4,153  ✓               71  ✗
Actual Malware             65  ✗            5,748  ✓

136 total misclassifications out of 10,037 samples.

Full design decisions and results documented in evaluation-and-design.md

---

CI/CD Pipeline

The project uses GitHub Actions for automated testing and deployment.

Workflow file: .github/workflows/ci-cd.yml

Trigger: Every push to main

Push to main
    │
    ▼
┌──────────────────────────────────────┐
│  Job: TEST  (ubuntu-latest, Py 3.12) │
│                                      │
│  1. Checkout code                    │
│  2. Install dependencies             │
│  3. Download dataset from Release    │
│     (data/brazilian-malware.csv)     │
│  4. Download model artifacts         │
│     (model/model.pkl, model/le.pkl)  │
│  5. Run pytest                       │
└──────────────────────────────────────┘

Why artifacts are hosted on GitHub Releases

The dataset (~50K rows) and trained model are too large or impractical to commit directly to the repo. They are hosted as GitHub Release assets and downloaded at test-time via curl:

• Dataset-v1.0 release → brazilian-malware.csv
• model-artifacts release → model.pkl, le.pkl

This keeps the repo lightweight while still allowing the CI pipeline to run tests against the real data and model.

Note: This project currently runs tests on push to main. A deploy job can be added by appending a second job with needs: test that triggers the Render deploy hook — deployment is then automatically blocked if any test fails.

To see pipeline runs: GitHub repo → Actions tab

---

Automated Tests

Three test files covering unit, integration, and smoke testing:

Unit Tests (tests/test_preprocessing.py)

pytest tests/test_preprocessing.py -v

• Model file loads correctly
• Predictions return correct shape
• Probabilities are between 0 and 1
• Predictions are binary (0 or 1 only)

Integration Tests (tests/test_app.py)

pytest tests/test_app.py -v

• GET / returns 200
• GET /health returns {"status": "ok"}
• POST /predict with a real sample returns Malware or Goodware

Smoke Test (tests/test_smoke.py)

APP_URL=https://malware-detection-mtvw.onrender.com pytest tests/test_smoke.py -v

• Hits the live /health endpoint to confirm successful deployment

Run all tests locally

pytest tests/ -v --ignore=tests/test_smoke.py

---

Deployment

The app is deployed on Render (free tier).

Live URL: See deployed.md

Adding auto-deploy to the pipeline

To add gated deployment, append this job to your ci-cd.yml:

  deploy:
    needs: test    # ← blocks deployment if tests fail
    runs-on: ubuntu-latest
    steps:
      - name: Trigger Render deployment
        run: curl -X POST "${{ secrets.RENDER_DEPLOY_HOOK }}"

Then add your Render deploy hook URL as a GitHub secret named RENDER_DEPLOY_HOOK.

---

File Descriptions

File	Purpose
config.py	Global constants — SEED=42, file paths
train.py	Complete training pipeline — run once to reproduce all results
eval.py	Loads model.pkl and evaluates on test_set.csv
app.py	Flask app — routes for /, /predict, /upload, /health
requirements.txt	Pinned Python dependencies
deployed.md	URL of the live deployed application
evaluation-and-design.md	CV results table, test metrics, design decisions
ai-tooling.md	AI tools used and how they were applied

---

Reproducing Results

All results are fully reproducible:

# 1. Install dependencies
pip install -r requirements.txt

# 2. Add dataset to data/brazilian-malware.csv

# 3. Run training (generates model.pkl and test_set.csv)
python train.py

# 4. Evaluate final model on test set
python eval.py

# 5. Run tests
pytest tests/ -v --ignore=tests/test_smoke.py

• Random seed SEED = 42 is set globally in config.py and passed to all models, splits, and CV folds
• All package versions are pinned in requirements.txt

---

AI Tooling

AI tools were used to accelerate development. See ai-tooling.md for details on what was used and how.

---

License

This project was built as part of the Quantic MSSE Introduction to Machine Learning course.