Skip to content

lastgenre: Genre spelling normalization (aliases) - #6466

Merged
JOJ0 merged 7 commits into
masterfrom
lastgenre_aliases
Jul 15, 2026
Merged

lastgenre: Genre spelling normalization (aliases)#6466
JOJ0 merged 7 commits into
masterfrom
lastgenre_aliases

Conversation

@JOJ0

@JOJ0 JOJ0 commented Mar 27, 2026

Copy link
Copy Markdown
Member

Description

This PR introduces a regex-based normalization (alias) system to unify variant genre tags and improves the plugin's documentation.

The normalization feature uses an ordered list of regular expression aliases to map variant spellings or synonyms to a single canonical name. The mapping keys act as re.Match.expand() templates, supporting \g<N> (or shorter, eg. \1) back-references to regex capture groups:

lastgenre:
    aliases:
        # Maps 'hip-hop' -> 'hip hop' using back-references 
        # (regex full-match)
        \1hop:
            - (hip|trip|jazz)y?[ /-]*hop
        
        # Static normalization examples
        drum and bass:
            - dnb
            - d&b
  • Where?: Integrated normalize_genre before is_ignored in both the LastFmClient (for clean lookup/filtering) and the core _resolve_genres loop (for uniform processing of existing file tags).
  • Updated default whitelist and genre tree files for canonical-name consistency, and curated the bundled aliases.yaml from the top 1,000 Last.fm tags to cover common inconsistencies without over-normalizing.
  • Any normalization that occurs is logged (extra debug log level beet -vvv ...)
  • Additional documentation changes:
    • Added a Choosing the Right Tool guide. This is for endusers but note that it might even be a primary starting point for maintainer review.
    • Added non-whitelist mode details to the Canonicalization chapter.
  • Additional unrelated changes:
    • Fixes a few variable name inconsistencies in the recently merged ignorelist features' tests.
    • Improved naming of ignorelist related type IgnorePatternsByArtist

@JOJ0
JOJ0 requested a review from a team as a code owner March 27, 2026 12:55
@github-actions

Copy link
Copy Markdown

Thank you for the PR! The changelog has not been updated, so here is a friendly reminder to check if you need to add an entry.

@JOJ0
JOJ0 marked this pull request as draft March 27, 2026 12:56
@codecov

codecov Bot commented Mar 27, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.13559% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.97%. Comparing base (dab3bd3) to head (c51f575).
⚠️ Report is 8 commits behind head on master.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
beetsplug/lastgenre/client.py 60.00% 4 Missing and 2 partials ⚠️
beetsplug/lastgenre/__init__.py 96.66% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6466      +/-   ##
==========================================
+ Coverage   74.91%   74.97%   +0.06%     
==========================================
  Files         163      163              
  Lines       21176    21209      +33     
  Branches     3342     3349       +7     
==========================================
+ Hits        15863    15902      +39     
+ Misses       4546     4538       -8     
- Partials      767      769       +2     
Files with missing lines Coverage Δ
beetsplug/lastgenre/utils.py 100.00% <100.00%> (ø)
beetsplug/lastgenre/__init__.py 82.62% <96.66%> (+0.94%) ⬆️
beetsplug/lastgenre/client.py 66.66% <60.00%> (+13.00%) ⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@JOJ0
JOJ0 force-pushed the lastgenre_aliases branch 2 times, most recently from 20b2d88 to 90064fa Compare March 28, 2026 22:52
@JOJ0
JOJ0 force-pushed the lastgenre_forbidden branch from 66a7d98 to 1785dab Compare March 28, 2026 23:12
@JOJ0 JOJ0 added lastgenre lastgenre plugin plugin Pull requests that are plugins related labels Mar 29, 2026
@JOJ0
JOJ0 force-pushed the lastgenre_forbidden branch from 1785dab to c5a14d3 Compare March 29, 2026 09:11
@JOJ0
JOJ0 force-pushed the lastgenre_aliases branch 4 times, most recently from d0cc527 to 12e7358 Compare March 29, 2026 10:09
@JOJ0
JOJ0 marked this pull request as ready for review March 29, 2026 10:17
@github-actions

Copy link
Copy Markdown

Thank you for the PR! The changelog has not been updated, so here is a friendly reminder to check if you need to add an entry.

@JOJ0

JOJ0 commented Mar 29, 2026

Copy link
Copy Markdown
Member Author

This is closely related to #6449 and might make sense to have a look when reviewing the former @snejus Many thanks in advance! 🙌

@JOJ0
JOJ0 force-pushed the lastgenre_aliases branch from 12e7358 to f4b9436 Compare March 29, 2026 11:03
Comment thread docs/plugins/lastgenre.rst
@JOJ0
JOJ0 force-pushed the lastgenre_forbidden branch 9 times, most recently from 87c272c to b12b314 Compare April 3, 2026 06:52
@JOJ0
JOJ0 force-pushed the lastgenre_aliases branch 3 times, most recently from afe7c06 to 000ff8f Compare May 22, 2026 16:06
@JOJ0

JOJ0 commented May 22, 2026

Copy link
Copy Markdown
Member Author

Hi @snejus I commented inline (sorry it's pretty messy here already :-/) addressed all suggestions and I am happy with the result. Please have a (hopefully final) look :-)

Note to myself: I'll probably simply squash all the commits or maybe reset and recommit everything so at least changes not directly involved with this new feature stay separate!

Comment thread beetsplug/lastgenre/aliases.yaml Outdated
Comment thread beetsplug/lastgenre/aliases.yaml Outdated
Comment thread beetsplug/lastgenre/aliases.yaml Outdated
Comment thread beetsplug/lastgenre/aliases.yaml Outdated
Comment thread beetsplug/lastgenre/aliases.yaml Outdated
Comment thread beetsplug/lastgenre/__init__.py Outdated
Comment thread beetsplug/lastgenre/utils.py Outdated
Comment thread beetsplug/lastgenre/utils.py Outdated
Comment thread beetsplug/lastgenre/client.py Outdated
Comment thread beetsplug/lastgenre/client.py Outdated
@JOJ0
JOJ0 force-pushed the lastgenre_aliases branch 5 times, most recently from 9ec1221 to 21908b9 Compare June 29, 2026 07:21
Comment thread beetsplug/lastgenre/genres-tree.yaml
@JOJ0

JOJ0 commented Jul 2, 2026

Copy link
Copy Markdown
Member Author

@snejus adressed all the code suggestions. That part is ready to merge I think.

What do you think of this change? 1ccdc2d

What should we do about alternative and minimal? Open for any suggestions as long as we finally get this merged :-)

Anything else left to do? You might want to review the changelog. Tried to explain what the goal was with default aliases and genre changes.

What do you think of this future feature ideas? It might address the real issues here:

  • Allow a combination of multiple genre tags to be combined: If something is tagged on last.fm as alternative and rock -> alternative rock, Same example with minimal/techno. A kind of "rule system".

And another one, which could make life easier for the user? Not sure:

  • Allow extending the default list, instead of just replacing it?

@JOJ0

JOJ0 commented Jul 5, 2026

Copy link
Copy Markdown
Member Author

@snejus regarding alternative and minimal, what do you think of this solution: 9db9704

if we don't put them into the tree at all but allow them in the whitelist. they simply stay as a valid genre but wont get canonicalized (given that wl and canonical are enabled)

@snejus

snejus commented Jul 6, 2026

Copy link
Copy Markdown
Member

What do you think of this change? 1ccdc2d

Seems like breaks is now gone from genres.txt?

@snejus

snejus commented Jul 6, 2026

Copy link
Copy Markdown
Member

Thanks, I had another look.

Regarding alternative and minimal, I think the solution in 9db9704 is exactly what I had in mind: don't translate them, but keep them in the whitelist and out of the tree. This way they remain valid genres even when both the whitelist and canonicalization are enabled, but we don't assign them a more specific meaning. Users can still add the translation themselves if they want to treat them as alternative rock and minimal techno.

There are still three failing tests, by the way: chillchillout, altalternative rock and alternativealternative rock. Would you mind updating or removing those cases?

The changelog makes sense to me. However, I'd probably avoid saying that the files were "audited" and that long-standing mismatches were "resolved" since this sounds a bit too definitive. I'd simply say that the default aliases normalize common Last.fm spelling and naming variants and that the whitelist and genre tree were updated with additional common Last.fm tags.

Regarding the future features:

Allow a combination of multiple genre tags to be combined

Makes sense! alternative + rock gives us the context which is missing when looking at alternative alone. On the other hand, I think this should be a separate feature since it needs to operate on the whole list of tags. We would need to decide whether matched tags are removed or retained and in which order the rules are applied.

Allow extending the default list, instead of just replacing it?

This also makes sense to me and is probably simpler. Maybe a separate aliases_add option would make the distinction between extending and replacing explicit?

I don't think either of these needs to be part of this PR.

@JOJ0

JOJ0 commented Jul 10, 2026

Copy link
Copy Markdown
Member Author

Hi @snejus thanks for the last feedback on future ideas (yes future as in separate feature requests), and for the finalization: As we discussed internally, let's keep everything as minimal and limited to spelling correction / and obvious duplication fixes only. Please go ahead and push a commit if you find the time. I'll fix all tests once that's done.

@snejus

snejus commented Jul 10, 2026

Copy link
Copy Markdown
Member

See #6823

JOJ0 and others added 7 commits July 15, 2026 16:04
- Genre alias normalization feature implementation
- Extend and simplify _filter_valid()
- Remove drop_ignored_genres()
- More smaller ignorelist related refactoring
client and main plugin classes use the same names for instance
variables now
Co-authored-by: Šarūnas Nejus <snejus@protonmail.com>
even though it's indirectly tested via resolve_genres tests and
elsewhere, it makes sense to keep them to help with future refactoring.
- Document the genre alias normalization feature and the default aliases
  we ship with the plugin (by using rst "literalinclude" referring to
  aliases.yaml directly)
- Clarify what canonicalization acutually does when used without a
  whitelist enabled.
- New docs chapter "Using the right tool"
- Fixes some inconsistencies in spelling in the data files we ship with
  the plugin as well as adds some new genres to both files that seem to
  be returned often by last.fm.
  - For some changes the default aliases feature helps to streamline
    inconsistencies that come from last.fm already.
- The default aliases file that ships with beets
  - Tests fixes that prove that most of the alias normalization patterns
    work as they are supposed to.

Co-authored-by: Šarūnas Nejus <snejus@protonmail.com>
@JOJ0

JOJ0 commented Jul 15, 2026

Copy link
Copy Markdown
Member Author

See #6823

Wohoo! It's merged! Thanks for the support! ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lastgenre lastgenre plugin

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants