Skip to content

feat: steer lineages by shared subject, not plausibility alone - #54

Merged
hedmana merged 4 commits into
mainfrom
feat/content-weighting
Sep 29, 2026
Merged

hedmana merged 4 commits into
mainfrom
feat/content-weighting

Conversation

@hedmana

@hedmana hedmana commented Sep 29, 2026

Copy link
Copy Markdown
Owner

Closes #53

What

  • gnn.content builds TF-IDF vectors over titles and abstracts, with plain English stopwords only, so no field vocabulary lives in src/. It measures how much two works share in subject. No model to train and no new dependency.
  • learned_flow takes an optional content. Each step's weight is the model's plausibility times (0.02 + similarity) ** 0.5: square-rooted, so content tempers the model rather than overruling it, and floored, so a work with no abstract on record still gets a share.
  • The API builds the vectors once per field, next to the embedding. Loading takes 0.8 s and a ranking 0.15 s.
  • gnn.lineages judges the served ranking beside a model_only ablation.

Why

In the citation graph a tool looks like any other hub, and the link model finds citations of it plausible because every paper makes them. What separates Adam from Bahdanau for GAT is what they're about.

The weighting was chosen on the dev papers only, from three strengths, and every other choice was frozen before the test papers were read:

dev found dev Recall@10 / nDCG@10 test found test Recall@10 / nDCG@10
learned, with content (served) 23/32 0.903 / 0.717 17/31 0.626 / 0.556
model only (before) 22/32 0.806 / 0.650 16/31 0.523 / 0.340
time_decayed_pagerank 21/32 0.778 / 0.567 14/31 0.474 / 0.303

The gain holds on the held-out papers, and is larger there: nDCG rises from 0.34 to 0.56. Stability is unchanged or better. The learned top 10 overlaps 94 to 97% across seeds, and 92 to 93% with 10% of citations dropped.

Tried on dev and left out:

  • Walk damping, age half-life and sharpening the model's weights moved a hit either way, which is noise at 12 papers.
  • Semantic Scholar's influence rate per work doesn't separate anything: Adam's is higher than ChebNet's.
  • Walk weights from a model of Semantic Scholar's per-citation influence labels tie content weighting and add nothing on top of it, at the cost of an extra stage and a rate-limited API.

Known cost: ChebNet's OpenAlex record carries another work's title and abstract, so content weighting now reads it as off-topic, and it leaves GCN's top six. A metadata correction for that record comes next.

Verify

uv run pytest tests/test_content.py tests/test_originators.py tests/test_lineages.py tests/test_api.py
uv run python -m gnn.lineages --field gnn   # about 4 minutes

Restart the API to serve the new ranking.

Generated with Claude Code

@hedmana
hedmana merged commit c599ff8 into main Sep 29, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tools outrank the works papers were built on

1 participant