GraphSAGE - PPI

A reproduction of "Inductive Representation Learning on Large Graphs" (Hamilton, Ying, Leskovec, NeurIPS 2017) on the public PPI (protein-protein interaction) benchmark. Algorithm 1/2, the four aggregator formulas (mean, GCN, LSTM, pooling), the unsupervised skip-gram loss, and the K=2/S1=25/S2=10 sampling architecture are implemented exactly as the paper specifies. The paper's own experiments run on 4x Titan X Pascal GPUs across three datasets; this repository targets the PPI dataset only, on a single consumer GPU (RTX 2060 SUPER). The full reproduction pipeline described below completes in about 75 minutes on that GPU.

Dataset

This reproduction runs on the public PPI (protein-protein interaction) dataset released by the GraphSAGE authors themselves, at http://snap.stanford.edu/graphsage/ppi.zip, the same file used for the paper's own PPI results (Table 1, Figure 2B, Figure 3). Nothing here is a resampled, filtered, or otherwise modified copy: scripts/run_ppi_experiments.py downloads and unzips this exact archive into data/ppi/ automatically (see Reproduction below).

Graphs	24 (20 train / 2 val / 2 test; cross-graph generalization)
Nodes	56,944 total, across all 24 graphs
Edges	818,716
Labels	121 Gene Ontology terms (multi-label: a protein can have several at once)
Features	50-dim (positional/motif gene sets + immunological signatures)
Feature sparsity	42% of nodes have no non-zero features

Results

Name	Unsup. F1 (reproduced)	Unsup. F1 (paper)	Sup. F1 (reproduced)	Sup. F1 (paper)
Random	0.398	0.396	0.398	0.396
Raw features	0.432	0.422	0.432	0.422
GraphSAGE-GCN	0.456	0.465	0.521	0.500
GraphSAGE-mean	0.459	0.486	0.585	0.598
GraphSAGE-LSTM	0.448	0.482	0.599	0.612
GraphSAGE-pool	0.473	0.502	0.602	0.600

This reproduction lands on the same qualitative ordering as the paper: GraphSAGE clearly beats raw features, which beats random, and GraphSAGE-pool is the strongest aggregator on both objectives. GraphSAGE-GCN is weakest in the supervised setting; in the unsupervised setting GraphSAGE-LSTM is weakest instead. The supervised score for each aggregator is the test F1 of the best (learning rate, model size) found by scripts/sweep_ppi_experiments.py's hyperparameter sweep; the unsupervised score is the test F1 of each aggregator's default single-configuration run, evaluated with a logistic-regression probe on the frozen embeddings (scripts/eval_unsupervised.py), not of the sweep. This table is results/ppi_results.md, regenerated by scripts/compile_results.py.

Reproduction pipeline

scripts/run_ppi_experiments_scaled.py runs the pipeline that produced the Results table above. It calls four scripts in sequence, each skipping any configuration that is already trained:

1. sweep_ppi_experiments.py: for each aggregator, trains {supervised, unsupervised} at learning rates {0.01, 0.001} (supervised) or {2e-6, 2e-7} (unsupervised), at "small" model size, and keeps whichever scores best on the validation split. Supervised runs for 10 epochs; unsupervised runs are capped at 3,000 of the ~17,050 steps in one epoch, used only to rank sweep candidates (the unsupervised score in the Results table above comes from each aggregator's separate default run, not from this sweep; see Results above). Writes results/best_hparams_ppi.json.
2. multiseed_ppi_experiments.py: retrains each of the 4 aggregators x {supervised, unsupervised}, using the sweep's selected hyperparameters, under a second random seed (124, alongside the existing seed 123), and reports mean +/- std F1. Supervised: 5 epochs. Unsupervised: capped at 3,000 steps. Writes results/seed_variance_ppi.json.
3. sensitivity_ppi_experiments.py: Section 4.3's two sweeps on graphsage_mean supervised, 5 epochs each: search depth K in {1, 2, 3} (S1=25, S2=S3=10), and neighborhood sample size S1=S2=S in {5, 10, 25, 50, 75} at fixed K=2. Writes results/sensitivity_ppi.json.
4. noise_robustness_ppi_experiments.py: GraphSAGE-GCN and GraphSAGE-pool, supervised, 5 epochs each, retrained from scratch with a fraction of node features replaced by Gaussian noise at levels {0.0, 0.5, 1.0}. Writes results/noise_robustness_ppi.json.

.venv\Scripts\python.exe scripts\run_ppi_experiments_scaled.py --dry_run
.venv\Scripts\python.exe scripts\run_ppi_experiments_scaled.py --time_budget_minutes 100

On a single RTX 2060 SUPER GPU, all four stages together take about 75 minutes. --time_budget_minutes stops the pipeline before starting a stage that would exceed the given number of minutes, logging whatever was skipped to results/scaled_run_manifest.json. Each of the four scripts also accepts --dry_run on its own, to print the commands it would run without executing them.

K-sensitivity: K=2 clearly beats K=1 (0.585 vs. 0.472 F1); K=3 does not improve on K=2 (0.575), matching the paper's "marginal returns beyond K=2" claim.

Sample-size sensitivity: F1 rises from 0.536 (S=5) to 0.640 (S=75), with most of the gain already reached by S=25 (0.593), while training time keeps climbing across the same range (0.35 to 0.83 sec/iteration).

Noise robustness: GraphSAGE-pool stays above both GraphSAGE-GCN and the raw-features baseline at every noise level (0.546 / 0.453 / 0.462 F1 at noise 0.0 / 0.5 / 1.0). GraphSAGE-GCN starts ahead of raw features at zero noise (0.490 vs. 0.432) but falls below it at full noise (0.389 vs. 0.393), exactly the structural-reliance gap Theorem 1 predicts for the pooling aggregator.

Repository structure

GraphSAGE.pdf                                      the paper
notebook.ipynb                                     paper walkthrough + real reproduced results
plotting.py                                        every matplotlib figure notebook.ipynb draws
requirements.txt

src/graphsage/                                     GraphSAGE implementation
  models.py                                        Algorithm 1/2 (sample + aggregate), unsupervised loss
  aggregators.py                                   AGGREGATE_k variants: mean, GCN, pool, LSTM
  supervised_models.py                             supervised classification head on top of models.py
  minibatch.py                                     graph -> padded-adjacency minibatches
  neigh_samplers.py                                fixed-size neighborhood sampling
  prediction.py                                    skip-gram link-prediction loss (Eq. 1)
  layers.py, inits.py, utils.py                    supporting building blocks
  supervised_train.py, unsupervised_train.py       CLI training entry points

scripts/                                           Reproduction pipeline
  baseline_ppi.py                                  Random + Raw features baselines
  eval_unsupervised.py                             logistic-regression eval of unsupervised embeddings (--split val/test)
  compile_results.py                               collects everything into results/ppi_results.md
  run_ppi_experiments.py                           end to end run, single default hyperparameter setting
  run_ppi_experiments_scaled.py                    runs the pipeline that produced the Results table above
  sweep_ppi_experiments.py                         learning-rate x model-size sweep, selected on validation F1
  multiseed_ppi_experiments.py                     repeats each variant over several --seed values, reports mean +/- std
  sensitivity_ppi_experiments.py                   Section 4.3 K / neighborhood-sample-size sweep on PPI
  noise_robustness_ppi_experiments.py              Figure 3 / Theorem 1 feature-noise-robustness check on PPI

data/ppi/                                          PPI dataset, from http://snap.stanford.edu/graphsage/ppi.zip
logs/                                              training logs, TensorBoard events, metrics.csv, saved embeddings + snapshots (not tracked)
results/                                           final metrics + figures, all produced by the scripts above
illustrations/                                     static, purely didactic diagrams

Reproduction

data/, logs/ and results/ all ship empty (git-ignored, placeholder-only); running the command below populates all three from scratch.

.venv\Scripts\python.exe scripts\run_ppi_experiments.py

This single script downloads the PPI dataset (if not already present), computes the Random/Raw-features baselines, runs all 4 aggregators x {supervised, unsupervised} on PPI at the code's single default hyperparameter setting, evaluates the unsupervised embeddings, compiles results/ppi_results.md, and finally re-executes notebook.ipynb in place. Its output also seeds run_ppi_experiments_scaled.py above, which is what actually produced the Results table.

Every unsupervised run is also passed --embedding_snapshot_steps, so each one produces both its final embeddings (for the F1 table above) and the intermediate checkpoints notebook.ipynb's PCA/t-SNE training-progression visuals need. The flag is generic, so this works for all four aggregators; notebook.ipynb section 8.2 lets you pick which one to look at via its EMBED_MODEL setting, defaulting to the best unsupervised performer. Every run, supervised or unsupervised, also writes a structured metrics.csv (step, epoch, loss, F1/MRR) into its log directory alongside the console output, which is what the notebook's training-dynamics charts read from.

What each script under the hood actually checks

run_ppi_experiments_scaled.py calls four scripts, each covering one piece of the paper this reproduction addresses:

• sweep_ppi_experiments.py: writes results/best_hparams_ppi.json (winning learning rate/model size and its validation + test F1, per aggregator/objective). compile_results.py and notebook.ipynb section 8 read the supervised half of this file directly; the unsupervised sweep winner's test score is instead recorded separately in results/eval_unsup_sweep_<model>.json, not otherwise used by the Results table (see Results above).
• multiseed_ppi_experiments.py: repeats a variant under a second --seed and reports mean/std. Writes results/seed_variance_ppi.json.
• sensitivity_ppi_experiments.py: writes results/sensitivity_ppi.json (K-sweep and sample-size sweep results), read by notebook.ipynb section 8.3. --k_only restricts a run to just the K half.
• noise_robustness_ppi_experiments.py: writes results/noise_robustness_ppi.json, read by notebook.ipynb section 9.1.

Each script accepts --dry_run and skips any configuration it has already trained, so re-running run_ppi_experiments_scaled.py after an interruption resumes rather than redoing finished work. See each script's own docstring for its exact methodology and the modeling choices made where the paper doesn't fully specify its own procedure (e.g. Figure 3's noise injection, or what S3 to use for K=3).