Parse secondary album types from MusicBrainz #2200

sepo · 2016-09-19T21:50:52Z

Problem

I've different radio plays. If i import one (e.g. https://musicbrainz.org/release-group/19e6209b-2ddc-30b8-9273-484bd075fe7b) to my library the ALBUMTYPE tag is always "other". The Musicbrainz site says "other + spokenword". I think the ALBUMTYPE should be "spokenword" instead. "other" is pretty useless.

Setup

OS: Fedora 24
Python version: 2.7.12
beets version: 1.3.19
Turning off plugins made problem go away (yes/no): no

My configuration (output of beet config) is:

directory: /mnt/Dateien/Musik
plugins: inline discogs chroma copyartifacts info badfiles edit scrub missing mbsync
paths:
    albumtype:soundtrack: Soundtracks/$album/$disc_and_track $title
    albumtype:audiobook: Hörbuch/$album/$disc_and_track $title
    albumtype:spokenword: Hörspiel/$albumartist/$album/$disc_and_track $title
    singleton:true: Tracks/$artist - $title
    comp:true: Compilations/$album%aunique{}/$disc_and_track $title
    default: Albums/$albumartist/$album%aunique{}/$disc_and_track $title
import:
    languages: de en
    detail: no
match:
    preferred:
        countries: ['DE', 'US', 'GB|UK']
        media: ['CD', 'Digital Media|File']
copyartifacts:
    extensions: .cue .CUE .LOG  .log .txt .TXT .jpeg .JPG .jpg .png .PNG
    print_ignored: yes
scrub:
    auto: yes
item_fields:
    disc_and_track: u'%02i.%02i' % (disc, track) if
                    disctotal > 1 else u'%02i' % (track)
edit:
    itemfields: album albumartist artist asin catalognum comments composer country day disc disc_and_track disctitle disctotal genre label language lyrics media month original_day original_month original_year title track tracktotal year    albumfields: album albumartist albumstatus albumtotal  
    albumtype asin catalognum country day disctotal genre label language month original_day original_month original_year year

The text was updated successfully, but these errors were encountered:

sampsyo · 2016-09-21T02:08:49Z

Thanks for the report! It looks like release groups are now allowed to have "secondary" types. See the XML here: https://musicbrainz.org/ws/2/release-group/19e6209b-2ddc-30b8-9273-484bd075fe7b

We currently only parse the primary type, but we should consider getting the rest of them too.

anshuman73 · 2016-11-28T17:44:52Z

Hi !
I'd like to work on this issue.
Can you help me get started ? (a GCI student here)

We want both the primary tag and the secondary tag to be displayed or check if the primary is a non-useful tag (for eg, 'other') and accordingly use the secondary tag ?

(P.S. I'm unable to claim tasks at the moment due to parental consents issue, should be able to soon)

sampsyo · 2016-11-28T18:23:05Z

Hi, @anshuman73! Great!

Let's start by using all the album types. We can just just join them all into a single string using a comma or a semicolon for now. That should work well for a first implementation, and we can consider more nuanced policies in a later phase.

anshuman73 · 2016-11-28T18:25:04Z

Sure, also, does beets use a local musicbrainz server or uses the api via http ?
I couldn't find the code where the requests to musicbrainz is made.
Any specific file you can point me to ?

sampsyo · 2016-11-28T18:33:41Z

There's no local server; it uses the public API.

The code in question here is beetsplug/mbcollection.py. You'll see the API calls go though the python-musicbrainzngs library, which is linked above.

anshuman73 · 2016-11-28T18:50:20Z

oh, thanks !
I'll get to work right away.

sampsyo · 2016-11-28T19:05:37Z

@anshuman73 Ack! I answered that question incorrectly—I was thinking of a different thread.

The actually relevant code is in beets/autotag/mb.py. That's where we pull data out of the MB responses and format strings for metadata fields.

Sorry about that! 😳 Please disregard my earlier comment.

anshuman73 · 2016-11-28T19:08:46Z

@sampsyo No issues.

I believe this is where I'd be focusing at ?

https://github.com/beetbox/beets/blob/master/beets/autotag/mb.py#L280

sampsyo · 2016-11-28T19:09:18Z

Yep, exactly!

anshuman73 · 2016-11-28T19:10:05Z

Also, is there any place where I can see example json/dictionary data given out ?
just so I can figure out where the secondary tags are stored (as the value of what key)

anshuman73 · 2016-11-28T19:11:36Z

Comparing it to the xml example given above, I cannot find a tag named "type", so I'm assuming it defaults to the primary-type tag ?

sampsyo · 2016-11-28T19:18:10Z

There's no place where it currently gets printed out, so I'd suggest temporarily inserting a print call to output the JSON data.

If you're interested in seeing (and experimenting with) how the XML is parsed precisely, the py-mb-ngs code might be the right place to look: https://github.com/alastair/python-musicbrainzngs/blob/master/musicbrainzngs/mbxml.py#L477

anshuman73 · 2016-11-28T19:29:42Z

Ahh that's just perfect.
Discloses the whole schema

anshuman73 · 2016-11-28T19:46:37Z

@sampsyo I think I've written the required code, I just can't figure out how to pass a release result to test the function.
Any suggestions ?

sampsyo · 2016-11-28T19:49:14Z

Sure! The test input is defined here:

beets/test/test_mb.py

Line 31 in 9de27c6

release = {

You can add a test method to that class to demonstrate the new data extraction.

anshuman73 · 2016-11-28T19:53:13Z

hmm, well it doesn't exactly have the primary type and secondary type fields. I just wanted to confirm the key names and whether secondary types are stored as a list or not. (though I checked the same from https://github.com/alastair/python-musicbrainzngs/blob/master/musicbrainzngs/mbxml.py)

sampsyo · 2016-11-28T19:55:18Z

Aha! Yes, you'll want to try running beets so it executes a real query in that case—and then check the data that gets returned by adding a print call.

anshuman73 · 2016-11-28T19:56:35Z

Hmm, I'll build it from my local source by simply executing the setup.py ?

sampsyo · 2016-11-28T19:57:58Z

Good question! This wiki page has instructions for getting set up with a local installation:
https://github.com/beetbox/beets/wiki/Hacking

In particular, I'd recommend pip install -e .. (It's sometimes convenient to use the --user flag too.)

anshuman73 · 2016-11-28T20:01:50Z

o.O I never knew about such an easy pip method existing for local installation.
Thanks !

anshuman73 · 2016-11-28T20:22:18Z

Yay ! Everything's copacetic
Submitting a PR in a min..

mineo · 2016-11-30T20:34:19Z

Since this (and #2294) have seen way more attention than my old #357, I'll just mention it here so it can get closed or something when this gets done.

sampsyo · 2016-11-30T21:54:56Z

Arg! Thanks for the reminder, @mineo—we should have noticed that a long time ago. As you suggested back on that ticket, the right way to resolve this is probably to store the types in separate tags.

anshuman73 · 2016-12-01T01:16:30Z

@sampsyo
So would you suggest me doing that ^ in the #2294 PR or add that when I'm tackling #1547 .
I think it would be better to implement it while adding support for more fields (now that we've already logged the new types)

sampsyo · 2016-12-01T04:18:05Z

Good question. Since it has to do with which fields we set on the imported items, let's worry about it after #1547.

Fixed #2200 and a small typo

sampsyo · 2017-06-11T00:55:36Z

I just merged #2294, which should log the multiple album types to beets's debug output. If you're interested in finishing off this feature, please try importing a few albums with that debug output enabled to check what's going on with the difference between the current one-shot album type and the new data—that will help us decide how to handle the new album types.

ghost · 2019-05-02T01:01:21Z

My audiobooks are still being imported as albumtype "other" on the latest linuxserver release.

squigglezworth · 2019-08-16T08:42:26Z

Is there any plan on finishing this? Or is there any way of using the secondary types in a path format?

tweitzel · 2019-09-22T05:43:16Z

Oh hey you still need some samples, I think. Here's the file that led me to this issue number.

pear-2:autotag xarph$ beet -v import /Volumes/books/Calibre\ Library/David\ L.\ Craddock/Dungeon\ Hacks_\ How\ NetHack\,\ Angband\,\ and\ Other\ Roguelikes\ Changed\ the\ Course\ of\ Video\ Games\ \(7\)/Dungeon\ Hacks_\ How\ NetHack\,\ Angband\,\ and\ O\ -\ David\ L.\ Craddock.m4b 
user configuration: /Users/xarph/.config/beets/config.yaml
data directory: /Users/xarph/.config/beets
plugin paths: /Users/xarph/work/whatlastgenre/plugin/beets/beetsplug
Sending event: pluginload
lastgenre: Loading canonicalization tree /usr/local/lib/python3.7/site-packages/beets-1.5.0-py3.7.egg/beetsplug/lastgenre/genres-tree.yaml
inline: adding item field disc_and_track
library database: /Users/xarph/Music/beetslibrary.blb
library directory: /Users/xarph/Music/beets
Sending event: library_opened
Sending event: import_begin
Sending event: import_task_created
Sending event: import_task_start
Looking up: /Volumes/books/Calibre Library/David L. Craddock/Dungeon Hacks_ How NetHack, Angband, and Other Roguelikes Changed the Course of Video Games (7)/Dungeon Hacks_ How NetHack, Angband, and O - David L. Craddock.m4b
Tagging David L. Craddock read by Mike Rylander - Dungeon Hacks: How NetHack, Angband, and Other Roguelikes Changed the Course of Video Games
No album ID found.
Search terms: David L. Craddock read by Mike Rylander - Dungeon Hacks: How NetHack, Angband, and Other Roguelikes Changed the Course of Video Games
Album might be VA: False
Searching for MusicBrainz releases with: {'release': 'dungeon hacks: how nethack, angband, and other roguelikes changed the course of video games', 'artist': 'david l. craddock read by mike rylander', 'tracks': '1'}
Requesting MusicBrainz release 0886f9d5-15e8-4c1e-a2f4-de6784f24c04
primary MB release type: other
secondary MB release type(s): audiobook
Sending event: albuminfo_received
Candidate: David L. Craddock read by Mike Rylander - Dungeon Hacks: How NetHack, Angband, and Other Roguelikes Changed the Course of Video Games (0886f9d5-15e8-4c1e-a2f4-de6784f24c04)
Computing track assignment...
...done.
Success. Distance: 0.73
Requesting MusicBrainz release 8ef21a4c-43b1-48ea-94f3-149473b5f3be
primary MB release type: other
secondary MB release type(s): audiobook
Sending event: albuminfo_received
Candidate: David L. Craddock read by Mike Rylander - Stay Awhile and Listen: How Two Blizzards Unleashed Diablo and Forged a Video-Game Empire (8ef21a4c-43b1-48ea-94f3-149473b5f3be)
Computing track assignment...
...done.
Success. Distance: 0.86
Requesting MusicBrainz release 2628ed19-d337-49ac-9fa1-4b468fe32329
primary MB release type: album
secondary MB release type(s): spokenword, audiobook
Sending event: albuminfo_received
Candidate: Agatha Christie read by David Suchet - How does your garden grow? and other stories (2628ed19-d337-49ac-9fa1-4b468fe32329)
Computing track assignment...
...done.
Success. Distance: 0.91
Requesting MusicBrainz release 2c5bc106-7cce-43b9-ae35-ecb612a3d8e9
primary MB release type: album
secondary MB release type(s): mixtape/street
Sending event: albuminfo_received
Candidate: David Banner - Sex, Drugs & Video Games (2c5bc106-7cce-43b9-ae35-ecb612a3d8e9)
Computing track assignment...
...done.
Success. Distance: 0.86
Requesting MusicBrainz release 68fb8ed6-0665-49dd-adc2-c8515b6e252e
primary MB release type: broadcast
secondary MB release type(s): interview
Sending event: albuminfo_received
Candidate: Tim Pritlove - NetHack (68fb8ed6-0665-49dd-adc2-c8515b6e252e)
Computing track assignment...
...done.
Success. Distance: 0.83
chroma: acoustid album candidates: 0
Evaluating 5 candidates.

/Volumes/books/Calibre Library/David L. Craddock/Dungeon Hacks_ How NetHack, Angband, and Other Roguelikes Changed the Course of Video Games (7)/Dungeon Hacks_ How NetHack, Angband, and O - David L. Craddock.m4b (1 items)
Sending event: before_choose_candidate
Finding tags for album "David L. Craddock read by Mike Rylander - Dungeon Hacks: How NetHack, Angband, and Other Roguelikes Changed the Course of Video Games".
Candidates:
1. David L. Craddock read by Mike Rylander - Dungeon Hacks: How NetHack, Angband, and Other Roguelikes Changed the Course of Video Games (26.6%) (missing tracks, tracks, media) (Digital Media, 2015)
2. Tim Pritlove - NetHack (17.2%) (album, artist, tracks, ...) (Digital Media, 2013, XW, Metaebene Personal Media, CRE199)
3. David Banner - Sex, Drugs & Video Games (14.5%) (missing tracks, album, artist, ...) (Digital Media, 2012, US)
4. David L. Craddock read by Mike Rylander - Stay Awhile and Listen: How Two Blizzards Unleashed Diablo and Forged a Video-Game Empire (13.7%) (missing tracks, album, tracks, ...) (Digital Media, 2016)
5. Agatha Christie read by David Suchet - How does your garden grow? and other stories (9.5%) (missing tracks, artist, album, ...) (3xCD, 2002, GB, HarperCollins Publishers UK, 0-00-714536-5)
# selection (default 1), Skip, Use as-is, as Tracks, Group albums,
Enter search, enter Id, aBort, Print tracks, plaY, eDit,
edit Candidates? s
Sending event: import_task_choice
Sending event: import
Sending event: cli_exit

tweitzel · 2019-09-22T05:48:23Z

If you add this functionality I'd love to be able to force an album type on command line, since audiobooks actually being tagged as such in musicbrainz is rather rare, it seems.

pear-2:autotag xarph$ beet -v import /Volumes/books/Calibre\ Library/Henri\ Charriere/Papillon\ \(171\)/Papillon\ -\ Henri\ Charriere.m4b 
user configuration: /Users/xarph/.config/beets/config.yaml
data directory: /Users/xarph/.config/beets
plugin paths: /Users/xarph/work/whatlastgenre/plugin/beets/beetsplug
Sending event: pluginload
lastgenre: Loading canonicalization tree /usr/local/lib/python3.7/site-packages/beets-1.5.0-py3.7.egg/beetsplug/lastgenre/genres-tree.yaml
inline: adding item field disc_and_track
library database: /Users/xarph/Music/beetslibrary.blb
library directory: /Users/xarph/Music/beets
Sending event: library_opened
Sending event: import_begin
Sending event: import_task_created
Sending event: import_task_start
Looking up: /Volumes/books/Calibre Library/Henri Charriere/Papillon (171)/Papillon - Henri Charriere.m4b
Tagging Henri Charriere - Papillon (Unabridged)
No album ID found.
Search terms: Henri Charriere - Papillon (Unabridged)
Album might be VA: False
Searching for MusicBrainz releases with: {'release': 'papillon (unabridged)', 'artist': 'henri charriere', 'tracks': '1'}
Requesting MusicBrainz release a891885c-ddd6-4ae4-9cab-22dd270e0b60
primary MB release type: other
secondary MB release type(s): audiobook
Sending event: albuminfo_received
Candidate: Henri Charrière gelesen von Oliver Rohrbeck - Papillon (a891885c-ddd6-4ae4-9cab-22dd270e0b60)
Computing track assignment...
...done.
Success. Distance: 0.90
Requesting MusicBrainz release e2804d68-83a5-4102-bbb2-ec6b9ecc7167
primary MB release type: album
Sending event: albuminfo_received
Candidate: Henri Dès - Henri Dès, Volume 16: L'Hirondelle et le Papillon (e2804d68-83a5-4102-bbb2-ec6b9ecc7167)
Computing track assignment...
...done.
Success. Distance: 0.86
Requesting MusicBrainz release 49f61758-4db9-4381-9e24-d0059994109b
primary MB release type: single
Sending event: albuminfo_received
Candidate: JACKSON - Papillon (49f61758-4db9-4381-9e24-d0059994109b)
Computing track assignment...
...done.
Success. Distance: 0.44
Requesting MusicBrainz release 3ad990f5-c054-4f82-b9b6-8f57d6fcb76b
primary MB release type: single
Sending event: albuminfo_received
Candidate: JACKSON - Papillon (3ad990f5-c054-4f82-b9b6-8f57d6fcb76b)
Computing track assignment...
...done.
Success. Distance: 0.48
Requesting MusicBrainz release 9e5613cc-d7d6-47c2-a9f0-41a9b77805e8
primary MB release type: album
secondary MB release type(s): live
Sending event: albuminfo_received
Candidate: SixMileBridge - Unabridged (9e5613cc-d7d6-47c2-a9f0-41a9b77805e8)
Computing track assignment...
...done.
Success. Distance: 0.85
chroma: acoustid album candidates: 0
Evaluating 5 candidates.

/Volumes/books/Calibre Library/Henri Charriere/Papillon (171)/Papillon - Henri Charriere.m4b (1 items)
Sending event: before_choose_candidate
Finding tags for album "Henri Charriere - Papillon (Unabridged)".
Candidates:
1. JACKSON - Papillon (56.3%) (artist, tracks, album, ...) (Digital Media, 2017, US, Team Wang Co., Ltd)
2. JACKSON - Papillon (51.6%) (artist, tracks, album, ...) (Digital Media, 2017, CN, Team Wang Co., Ltd)
3. SixMileBridge - Unabridged (15.2%) (missing tracks, artist, tracks, ...)
4. Henri Dès - Henri Dès, Volume 16: L'Hirondelle et le Papillon (13.9%) (missing tracks, album, artist, ...) (CD, 2008, Productions Mary-Josée)
5. Henri Charrière gelesen von Oliver Rohrbeck - Papillon (9.6%) (missing tracks, artist, tracks, ...) (Digital Media, 2011, DE, Lauscherlounge)
# selection (default 1), Skip, Use as-is, as Tracks, Group albums,
Enter search, enter Id, aBort, eDit, edit Candidates, Print tracks,
plaY?

sampsyo added the feature features we would like to implement label Sep 21, 2016

sampsyo changed the title ~~ALBUMTYPE is "other" instead of "spokenword" or "other + spokenword"~~ Parse secondary album types from MusicBrainz Sep 21, 2016

sampsyo closed this as completed in 318661b Jun 11, 2017

sampsyo added a commit that referenced this issue Jun 11, 2017

Merge pull request #2294 from anshuman73/master

290162a

Fixed #2200 and a small typo

sampsyo reopened this Jun 11, 2017

Andy2244 mentioned this issue Dec 20, 2017

missing albumartist translation and missing extra MUSICBRAINZ_ALBUMTYPE #2764

Closed

sampsyo mentioned this issue Jul 9, 2020

secondary album type overrides primary #3663

Closed

edgars-supe mentioned this issue Sep 8, 2021

Store secondary release types #4045

Merged

3 tasks

sampsyo closed this as completed in #4045 Sep 9, 2021

jeandeaual mentioned this issue Feb 19, 2022

Add new list fields set by MusicBrainz Picard beetbox/mediafile#60

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Parse secondary album types from MusicBrainz #2200

Parse secondary album types from MusicBrainz #2200

sepo commented Sep 19, 2016 •

edited

Loading

sampsyo commented Sep 21, 2016

anshuman73 commented Nov 28, 2016

sampsyo commented Nov 28, 2016

anshuman73 commented Nov 28, 2016

sampsyo commented Nov 28, 2016

anshuman73 commented Nov 28, 2016 •

edited

Loading

sampsyo commented Nov 28, 2016 •

edited

Loading

anshuman73 commented Nov 28, 2016

sampsyo commented Nov 28, 2016

anshuman73 commented Nov 28, 2016

anshuman73 commented Nov 28, 2016

sampsyo commented Nov 28, 2016

anshuman73 commented Nov 28, 2016

anshuman73 commented Nov 28, 2016

sampsyo commented Nov 28, 2016

anshuman73 commented Nov 28, 2016

sampsyo commented Nov 28, 2016

anshuman73 commented Nov 28, 2016

sampsyo commented Nov 28, 2016

anshuman73 commented Nov 28, 2016

anshuman73 commented Nov 28, 2016

mineo commented Nov 30, 2016

sampsyo commented Nov 30, 2016

anshuman73 commented Dec 1, 2016

sampsyo commented Dec 1, 2016

sampsyo commented Jun 11, 2017

ghost commented May 2, 2019

squigglezworth commented Aug 16, 2019

tweitzel commented Sep 22, 2019 •

edited

Loading

tweitzel commented Sep 22, 2019

Parse secondary album types from MusicBrainz #2200

Parse secondary album types from MusicBrainz #2200

Comments

sepo commented Sep 19, 2016 • edited Loading

Problem

Setup

sampsyo commented Sep 21, 2016

anshuman73 commented Nov 28, 2016

sampsyo commented Nov 28, 2016

anshuman73 commented Nov 28, 2016

sampsyo commented Nov 28, 2016

anshuman73 commented Nov 28, 2016 • edited Loading

sampsyo commented Nov 28, 2016 • edited Loading

anshuman73 commented Nov 28, 2016

sampsyo commented Nov 28, 2016

anshuman73 commented Nov 28, 2016

anshuman73 commented Nov 28, 2016

sampsyo commented Nov 28, 2016

anshuman73 commented Nov 28, 2016

anshuman73 commented Nov 28, 2016

sampsyo commented Nov 28, 2016

anshuman73 commented Nov 28, 2016

sampsyo commented Nov 28, 2016

anshuman73 commented Nov 28, 2016

sampsyo commented Nov 28, 2016

anshuman73 commented Nov 28, 2016

anshuman73 commented Nov 28, 2016

mineo commented Nov 30, 2016

sampsyo commented Nov 30, 2016

anshuman73 commented Dec 1, 2016

sampsyo commented Dec 1, 2016

sampsyo commented Jun 11, 2017

ghost commented May 2, 2019

squigglezworth commented Aug 16, 2019

tweitzel commented Sep 22, 2019 • edited Loading

tweitzel commented Sep 22, 2019

sepo commented Sep 19, 2016 •

edited

Loading

anshuman73 commented Nov 28, 2016 •

edited

Loading

sampsyo commented Nov 28, 2016 •

edited

Loading

tweitzel commented Sep 22, 2019 •

edited

Loading