Skip to content

fix: give training the patience to outlast its early popularity peak - #44

Merged
hedmana merged 2 commits into
mainfrom
fix/training-patience
Sep 29, 2026
Merged

hedmana merged 2 commits into
mainfrom
fix/training-patience

Conversation

@hedmana

@hedmana hedmana commented Sep 29, 2026

Copy link
Copy Markdown
Owner

Closes #43

What

  • TrainConfig defaults go from 400 epochs and patience 40 to 600 epochs and patience 100.
  • PLAN.md's expanded-snapshot results now come from the served snapshot, trained with these defaults.

Why

On the expanded snapshot, one step from a random start ranks by popularity alone, and validation AP reads 0.899. Learning more dips below that before rising past it: 0.86 at epoch 2, 0.90 at 50, 0.92 at 150, 0.93 at 300. With patience 40, training kept epoch 1 and stopped at 41. The served model was effectively untrained.

With the new defaults, gnn.evaluate on the served snapshot, paired over 15 runs against the logistic baseline:

uniform year-matched
MRR +0.024 ± 0.008, 15/15 +0.013 ± 0.003, 15/15
AUC +0.015 ± 0.002, 15/15 +0.030 ± 0.006, 15/15

The plan had recorded that the model only tied popularity under year-matched negatives. That came from the undertrained model. On the old snapshot, training already settled around epoch 100 and is unaffected.

Verify

uv run python -m gnn.train --field gnn        # best_epoch well past 1 in metrics.json
uv run python -m gnn.evaluate --field gnn     # about 8 minutes

Generated with Claude Code

@hedmana
hedmana merged commit a81c048 into main Sep 29, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Training stops at its first epoch on the expanded snapshot

1 participant