Repository navigation
fix: fold duplicate records of one work into one - #62
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #61
What
graph.build.cleanfolds duplicate records into one. It keeps the most cited record, together with every citation and any abstract the kept record lacks. The report counts these asmerged_nodes.Deep learningis both LeCun 2015 and Goodfellow 2016. A given name is too strict, because DiffPool's first author is both Rex and Zhitao Ying.duplicate_works, names records filed under another work's metadata. Ingnn.yamlthese are GCN, GraphSAGE, GIN, VGAE, Attention Is All You Need, Adam, BERT, Hamilton et al. 2017, APPNP (under its first author's later name), and a 2020 book chapter that OpenAlex credits with GAT's citations. Each one was identified by what its citers cite on Semantic Scholar.W2964015378moves fromexclude_workstoduplicate_works. Pitfalls and Neural Combinatorial Optimization get their titles back; NCO's authors are still wrong, since no author correction exists.W4292779060is excluded, because it has too few citers to identify.Why
About 450 crawled works sat in the graph twice, which split each work's citations and could rank a work twice. For example, ChebNet had 1446 + 435 citations inside the graph and DiffPool 415 + 414. Excluding GCN's mis-filed record discarded 2458 citations.
On the rebuilt snapshot
gnn-20260930, the graph goes from 10051 works to 9599, and GCN's citations inside the graph go from 2453 to 3779.This is a metadata fix, not tuning, and the ranking does not improve. The proxy gate still holds, with 15/15 paired MRR wins over logistic on both samplers (+0.023 uniform, +0.014 year-matched). Adam, no longer split across two records, enters GGSNN's top 5, which is the known problem of tools competing with originators.
Verify
uv run pytest tests/test_build.py uv run python -m graph.build --field gnn # merged_nodes 463 uv run python -m gnn.lineages --field gnnGenerated with Claude Code