discogs: Parse IDs from new release URL format #4080

jdheyburn · 2021-10-01T07:14:37Z

Problem

With some files located in directory at with no tags
With discogs plugin enabled

Running this command in verbose (-vv) mode:

$ beet -vv import at

Then entering i to enter release ID, for example: https://www.discogs.com/release/18377836-Aphex-Twin-Come-To-Daddy

Led to this problem:

Enter release ID: https://www.discogs.com/release/18377836-Aphex-Twin-Come-To-Daddy
Tagging  - 
Searching for album ID: https://www.discogs.com/release/18377836-Aphex-Twin-Come-To-Daddy
Requesting MusicBrainz release https://www.discogs.com/release/18377836-Aphex-Twin-Come-To-Daddy
Invalid MBID (https://www.discogs.com/release/18377836-Aphex-Twin-Come-To-Daddy).
discogs: Searching for release https://www.discogs.com/release/18377836-Aphex-Twin-Come-To-Daddy
Evaluating 0 candidates.
Sending event: before_choose_candidate
No matching release found for 4 tracks.
For help, see: https://beets.readthedocs.org/en/latest/faq.html#nomatch

Here's a link to the music files that trigger the bug (if relevant): N/A - any music files will do

Here is some verbose logging while beets was starting up:

 0.87
Sending event: albuminfo_received
Candidate: Skrewdriver - Boots And Braces (1654191)
Computing track assignment...
...done.
Success. Distance: 0.85
Sending event: albuminfo_received
Candidate: Saint Etienne - Continental (1778268)
Computing track assignment...
...done.
Success. Distance: 0.88
Sending event: albuminfo_received
Candidate: X-102 - Titan EP (1393615)
Computing track assignment...
...done.
Success. Distance: 0.83
Evaluating 5 candidates.

Note that the search option worked.

Enter search, enter Id, aBort, eDit, edit Candidates? e
Artist: Aphex Twin
Album: Come to daddy
Tagging  - 
No album ID found.
Search terms: Aphex Twin - Come to daddy
Album might be VA: False
Searching for MusicBrainz releases with: {'release': 'come to daddy', 'artist': 'aphex twin', 'tracks': '4'}
Requesting MusicBrainz release f7a09aff-6a8c-49c1-b8fb-4d2e93bebd6b
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
primary MB release type: single
Sending event: mb_album_extract
Sending event: albuminfo_received
Candidate: Aphex Twin - Come to Daddy (f7a09aff-6a8c-49c1-b8fb-4d2e93bebd6b)
Computing track assignment...
...done.
Success. Distance: 0.76
Requesting MusicBrainz release 9197ad55-4c0a-3cb3-b190-e1f9062b867c
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
primary MB release type: single
Sending event: mb_album_extract
Sending event: albuminfo_received
Candidate: Aphex Twin - Come to Daddy (9197ad55-4c0a-3cb3-b190-e1f9062b867c)
Computing track assignment...
...done.
Success. Distance: 0.76
Requesting MusicBrainz release 32ad4a8c-cd44-3637-ac39-3479d7be8fb2
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
primary MB release type: ep
Sending event: mb_album_extract
Sending event: albuminfo_received
Candidate: Aphex Twin - Come to Daddy (32ad4a8c-cd44-3637-ac39-3479d7be8fb2)
Computing track assignment...
...done.
Success. Distance: 0.81
Requesting MusicBrainz release c8d680de-3bbd-3546-bf0f-ac151e079c1b
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
primary MB release type: ep
Sending event: mb_album_extract
Sending event: albuminfo_received
Candidate: Aphex Twin - Come to Daddy (c8d680de-3bbd-3546-bf0f-ac151e079c1b)
Computing track assignment...
...done.
Success. Distance: 0.81
Requesting MusicBrainz release 0931f506-5138-3c27-b561-fc95f0d813a9
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
primary MB release type: ep
Sending event: mb_album_extract
Sending event: albuminfo_received
Candidate: Aphex Twin - Come to Daddy (0931f506-5138-3c27-b561-fc95f0d813a9)
Computing track assignment...
...done.
Success. Distance: 0.81
discogs: Searching for master release 27457
discogs: hit rate limit, waiting for 1.8842348098754882 seconds
discogs: Searching for master release 27457
discogs: hit rate limit, waiting for 2.157538557052612 seconds
discogs: Searching for master release 27457
discogs: hit rate limit, waiting for 2.085971975326538 seconds
discogs: Searching for master release 27457
discogs: hit rate limit, waiting for 1.992177629470825 seconds
discogs: Searching for master release 27457
discogs: hit rate limit, waiting for 2.0841585636138915 seconds
Sending event: albuminfo_received
Candidate: Aphex Twin - Come To Daddy (3662)
Computing track assignment...
...done.
Success. Distance: 0.69
Sending event: albuminfo_received
Candidate: Aphex Twin - Come To Daddy (3903368)
Computing track assignment...
...done.
Success. Distance: 0.69
Sending event: albuminfo_received
Candidate: Aphex Twin - Come To Daddy (6157)
Computing track assignment...
...done.
Success. Distance: 0.78
Sending event: albuminfo_received
Candidate: Aphex Twin - Come To Daddy (52731)
Computing track assignment...
...done.
Success. Distance: 0.78
Sending event: albuminfo_received
Candidate: Aphex Twin - Come To Daddy (10818482)
Computing track assignment...
...done.
Success. Distance: 0.69
Evaluating 10 candidates.
Sending event: before_choose_candidate
Finding tags for album " - ".
Candidates:
1. Aphex Twin - Come To Daddy (31.2%) (tracks, album, artist, ...) (Discogs, Vinyl, 1997, UK, Warp Records, WAP94)
2. Aphex Twin - Come To Daddy (31.2%) (tracks, album, artist, ...) (Discogs, Vinyl, 2012, UK, Warp Records, WAP94)
3. Aphex Twin - Come To Daddy (31.2%) (tracks, album, artist, ...) (Discogs, Vinyl, 2017, UK, Warp Records, WAP94)
4. Aphex Twin - Come to Daddy (23.8%) (tracks, album, artist) (CD, 1997, GB, Warp, WAP94CD)
5. Aphex Twin - Come to Daddy (23.8%) (tracks, album, artist) (12" Vinyl, 1997, GB, Warp, WAP94)
6. Aphex Twin - Come To Daddy (22.1%) (tracks, missing tracks, album, ...) (Discogs, CD, 1997, UK, Warp Records, WAP94CDX)
7. Aphex Twin - Come To Daddy (22.1%) (tracks, missing tracks, album, ...) (Discogs, CD, 1997, US, Sire, 31001)
8. Aphex Twin - Come to Daddy (18.9%) (tracks, missing tracks, album, ...) (CD, 1997, DE, Rough Trade, RTD126.3383.2)
9. Aphex Twin - Come to Daddy (18.9%) (tracks, missing tracks, album, ...) (CD, 1997, JP, Sire Records, WPCR-1603)
10. Aphex Twin - Come to Daddy (18.9%) (tracks, missing tracks, album, ...) (CD, 1997, US, Sire Records, 31001)

Setup

OS: PopOS
Python version: 3.9.5
beets version: 1.5.0
Turning off plugins made problem go away (yes/no): no

My configuration (output of beet config) is:

plugins: edit discogs inline info badfiles missing embedart fetchart
directory: /mnt/nfs/Backup/media/vinyl
library: /mnt/nfs/Backup/media/beets-db/beets-vinyl.db

import:
    move: no
    copy: yes
    timid: yes

paths:
    default: $albumartist/$album%aunique{}/%if{$multidisc,CD$disc/}$track $artist - $title
    comp: Compilations/$album%aunique{}/%if{$multidisc,CD$disc/}$track $artist - $title
    singleton: Singles/$artist - $title
item_fields:
    multidisc: 1 if disctotal > 1 and media == 'CD' else 0
original_date: yes
threaded: yes
fetchart:
    cover_names: cover
    minwidth: 500
    store_source: yes
    auto: yes
    maxwidth: 0
    quality: 0
    max_filesize: 0
    enforce_ratio: no
    cautious: no
    sources:
    - filesystem
    - coverart
    - itunes
    - amazon
    - albumart
    google_key: REDACTED
    google_engine: 001442825323518660753:hrh5ch1gjzm
    fanarttv_key: REDACTED
    lastfm_key: REDACTED
    high_resolution: no
embedart:
    maxwidth: 0
    auto: yes
    compare_threshold: 0
    ifempty: no
    remove_art_file: no
    quality: 0
pathfields: {}
album_fields: {}
missing:
    count: no
    total: no
    album: no
edit:
    albumfields: album albumartist
    itemfields: track title artist album
    ignore_fields: id path
discogs:
    apikey: REDACTED
    apisecret: REDACTED
    tokenfile: discogs_token.json
    source_weight: 0.5
    user_token: REDACTED
    separator: ', '
    index_tracks: no

I just tried setting my own discogs.apikey and discogs.apisecret and I received the same hit rate limit output as above.

Might be worth mentioning that this was working 2 days ago, I hadn't changed anything in my setup between then. My requirements.txt file is:

audioread==2.1.9
beautifulsoup4==4.9.3
beetcamp==0.8.0
beets==1.5.0
cached-property==1.5.2
certifi==2020.12.5
chardet==4.0.0
confuse==1.4.0
discogs-client==2.3.0
idna==2.10
isodate==0.6.0
jellyfish==0.8.2
mediafile==0.6.0
munkres==1.1.4
musicbrainzngs==0.7.1
mutagen==1.45.1
oauthlib==3.1.0
packaging==20.9
Pillow==8.1.0
pyacoustid==1.2.0
pycountry==20.7.3
pyparsing==2.4.7
PyYAML==5.4.1
requests==2.25.1
requests-oauthlib==1.3.0
six==1.15.0
soupsieve==2.2
Unidecode==1.2.0
urllib3==1.26.3

The text was updated successfully, but these errors were encountered:

sampsyo · 2021-10-01T11:36:20Z

It looks like discogs may have changed the way they format their URLs? As you can see, we only expect a number after the / in the URL, possibly followed by a bracket:

beets/beetsplug/discogs.py

Lines 212 to 213 in a187739

    
           match = re.search(r'(^|\[*r|discogs\.com/.+/release/)(\d+)($|\])', 
        
                             album_id)

What happens, for example, if you try pasting just https://www.discogs.com/release/18377836?

jdheyburn · 2021-10-01T13:03:16Z

That URL didn't work either, but you had me go to regex101 and check the regex being used vs what Discogs uses now [1] versus before [2] and you can see there is definitely a change. Both URLs still work so I guess the regex will need to be updated to account for both?

[1]
https://www.discogs.com/release/18377836-Aphex-Twin-Come-To-Daddy

[2]
https://www.discogs.com/Roy-Davis-Jr-Tha-Black-Light-Exposure-EP/release/19766479

jdheyburn · 2021-10-01T17:33:16Z

Just did a switcheroo of

https://www.discogs.com/release/15963324-Larry-Heard-Sceneries-Not-Songs-Volume-One

to

https://www.discogs.com/Larry-Heard-Sceneries-Not-Songs-Volume-One/release/15963324

and it was able to pick it up

Enter release ID: https://www.discogs.com/Larry-Heard-Sceneries-Not-Songs-Volume-One/release/15963324
Correcting tags from:
    (unknown album)
To:
    Larry Heard - Sceneries Not Songs, Volume One
URL:
    https://www.discogs.com/release/15963324-Larry-Heard-Sceneries-Not-Songs-Volume-One

sampsyo · 2021-10-01T22:16:17Z

Ah, very tricky! Looks like we should make our regex flexible enough to work for both styles of URLs… or else just write two different regexes.

Tested with https://www.discogs.com/SHOUSE-Love-Tonight-Robin-Schulz-Remix/release/20356324 https://www.discogs.com/release/20356324-SHOUSE-Love-Tonight-Robin-Schulz-Remix

Fix #4080

sampsyo added the needinfo We need more details or follow-up from the filer before this can be tagged "bug" or "feature." label Oct 1, 2021

sampsyo added feature features we would like to implement and removed needinfo We need more details or follow-up from the filer before this can be tagged "bug" or "feature." labels Oct 1, 2021

sampsyo changed the title ~~Discogs enter ID no longer works~~ discogs: Parse IDs from new release URL format Oct 1, 2021

arogl added a commit to arogl/beets that referenced this issue Oct 2, 2021

Fix beetbox#4080

c4f2fff

Tested with https://www.discogs.com/SHOUSE-Love-Tonight-Robin-Schulz-Remix/release/20356324 https://www.discogs.com/release/20356324-SHOUSE-Love-Tonight-Robin-Schulz-Remix

arogl mentioned this issue Oct 2, 2021

Fix #4080 #4085

Merged

3 tasks

wisp3rwind closed this as completed in #4085 Oct 5, 2021

wisp3rwind added a commit that referenced this issue Oct 5, 2021

Merge pull request #4085 from arogl/fix-4080

afbb12d

Fix #4080

sampsyo mentioned this issue Jan 7, 2022

discogs: discogs_albumid is always 0 #4225

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

discogs: Parse IDs from new release URL format #4080

discogs: Parse IDs from new release URL format #4080

jdheyburn commented Oct 1, 2021 •

edited

Loading

sampsyo commented Oct 1, 2021

jdheyburn commented Oct 1, 2021 •

edited

Loading

jdheyburn commented Oct 1, 2021

sampsyo commented Oct 1, 2021

discogs: Parse IDs from new release URL format #4080

discogs: Parse IDs from new release URL format #4080

Comments

jdheyburn commented Oct 1, 2021 • edited Loading

Problem

Setup

sampsyo commented Oct 1, 2021

jdheyburn commented Oct 1, 2021 • edited Loading

jdheyburn commented Oct 1, 2021

sampsyo commented Oct 1, 2021

jdheyburn commented Oct 1, 2021 •

edited

Loading

jdheyburn commented Oct 1, 2021 •

edited

Loading