Skip to content

Add backfill script to link subgenre Tags to parent genres - #45

Open
Chisomnwa wants to merge 1 commit into
Open-Book-Genome-Project:mainfrom
Chisomnwa:feature/backfill-genre-refs
Open

Add backfill script to link subgenre Tags to parent genres#45
Chisomnwa wants to merge 1 commit into
Open-Book-Genome-Project:mainfrom
Chisomnwa:feature/backfill-genre-refs

Conversation

@Chisomnwa

@Chisomnwa Chisomnwa commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

Add backfill script for genre refs on subgenre Tags

Summary

Adds scripts/backfill_genre_refs.py to populate the genre field on our 24 existing subgenre Tags, linking each subgenre to its parent genre Tag(s). This is step 2 of the Genre Explorer phasing plan (issue #13158).

What this does

Reads the parent genre mapping from our subgenre vocabulary (tag_types/subgenres/vocabulary.json), matches each subgenre to its parent genre Tag key from the genre vocabulary, and writes the genre reference to each subgenre Tag object on Open Library.

Example

Biopunk (/tags/OL268T) will get:

"genre": [{"key": "/tags/OL179T"}]

Linking it to Sci-Fi (/tags/OL179T).

Gothic (/tags/OL281T) will get:

"genre": [{"key": "/tags/OL171T"}, {"key": "/tags/OL177T"}, {"key": "/tags/OL174T"}]

Linking it to Horror, Romance, and Literary.

Usage

python scripts/backfill_genre_refs.py --dry-run   # preview only
python scripts/backfill_genre_refs.py              # write changes

Dry-run results:
backfill_genre_refs

  • Loaded 24 subgenre-parent mappings from vocabulary
  • Loaded 21 genre mappings from vocabulary
  • Done: 24 subgenre Tags updated, 0 errors

Why we load genre keys from vocabulary, not OL search API

We initially tried querying the OL search API (/search.json) to discover genre Tags, but Tags aren't indexed in Solr yet (@cdrini hasn't indexed them). The search API returns 0 results for Tags.

But, the script doesn't need Solr indexing at all — it uses the OL JSON API (via olclient) to fetch and save Tags, not the search API.

We load genre keys from the vocabulary files instead of querying OL because:

  • It's simpler — the vocabulary already has every genre Tag key we created in Phase 2A
  • No external dependency — no network calls for the lookup
  • Solr is irrelevant here — Solr only matters later, when the Genre Explorer needs to search/query the genre→subgenre hierarchy

When Tags get indexed in Solr, the search-based approach can be used as an alternative.

Prerequisites

The PR adding genre property to /type/tag must be merged first PR link
~/.config/ol.ini must have bot credentials (for non-dry-run)
Usage

Reviewrs

@mekarpeles

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant