beatport: Fetch musical key, BPM, and genre data #2080

itsbrex · 2016-06-23T17:26:14Z

Problem

Everything seems to be getting pulled in except BPM & Key. Genre tag appears to be hit or miss. All tracks are from beatport, so the tags should be there. Looking at the API docs I see the facets listed. Are these actually accessible in Track Info?

Requires catalog API Permission 

Content can be filtered on facets. Ex:facets=genreName:House, subGenreName:Hard 

Use returnFacets to obtain specific information about the return set.

Allowed facets/returnFacets:

Note: Not all facets are relevant to all content types.

artistName
performerName
artist
artistId
performerId
label
labelName
labelId
subGenre
subGenreName
subGenreId
genre
genreName
genreId
bpm         <------------
key          <------------
tagId

Setup

OS: Mac OS X 10.11.5 (15F34)
Python version: 2,7
beets version: 1.3.18
Turning off plugins made problem go away (yes/no): no

My configuration (output of beet config) is:

briansmac:beets anodigital$ beet config
lyrics:
    bing_lang_from: []
    bing_client_secret: REDACTED
    google_API_key: REDACTED
    force: no
    google_engine_ID: REDACTED
    sources:
    - google
    - lyricwiki
    - lyrics.com
    - musixmatch
    auto: yes
    fallback:
    genius_api_key: REDACTED
    bing_lang_to:
library: /Users/anodigital/Desktop/music.blb
original_date: yes
embedart:
    maxwidth: 1000
    compare_threshold: 0
    auto: yes
    ifempty: no
    remove_art_file: no
pluginpath: ~/github/beets/beetsplug

plugins: beatport fromfilename discogs embedart fetchart ftintitle fuzzy importadded lyrics mbsync spotify types
beatport:
    source_weight: 0.5
    tokenfile: beatport_token.json
    apikey: REDACTED
    apisecret: REDACTED
threaded: yes
art_filename: cover
duplicate_action: remove
none_rec_action: skip
ignore: .AppleDouble ._* *~ .DS_Store
directory: /Users/anodigital/Desktop/Music

import:
    copy: no
    write: yes
    resume: yes
    quiet_fallback: skip
    timid: no
    log: beetslog.txt
group_albums: no

match:
    strong_rec_thresh: 0.05
    medium_rec_thresh: 0.25
    rec_gap_thresh: 0.25
    max_rec:
        missing_tracks: medium
        unmatched_tracks: medium
    ignored: missing_tracks unmatched_tracks
discogs:
    source_weight: 0.5
    tokenfile: discogs_token.json
    apikey: REDACTED
    apisecret: REDACTED
fuzzy:
    threshold: 0.7
    prefix: '~'
spotify:
    regex: []
    artist_field: albumartist
    mode: list
    track_field: title
    album_field: album
    region_filter:
    show_failures: no
    tiebreak: popularity
importadded:
    preserve_write_mtimes: no
    preserve_mtimes: no
fetchart:
    auto: yes
    minwidth: 0
    sources:
    - filesystem
    - coverart
    - itunes
    - amazon
    - albumart
    google_engine: 001442825323518660753:hrh5ch1gjzm
    enforce_ratio: no
    cautious: no
    maxwidth: 0
    store_source: no
    google_key: REDACTED
    fanarttv_key: REDACTED
    cover_names:
    - cover
    - front
    - art
    - album
    - folder
ftintitle:
    auto: yes
    drop: no
    format: feat. {0}

The text was updated successfully, but these errors were encountered:

sampsyo · 2016-06-23T18:00:27Z

Yep, it looks like the plugin doesn't get those fields yet. Seems like a reasonable addition if you're interested.

itsbrex · 2016-06-23T18:06:11Z

@sampsyo definitely interested! 😄

jbaiter · 2016-06-28T18:07:33Z

I'd love to implement this, but can you tell me how I can pass this information to beets, @sampsyo?
I don't see a bpm or key field on the TrackInfo class.

As for the Genre tags, do you have a few IDs of releases that are missing genre tags so I can reproduce this?

sampsyo · 2016-06-28T18:40:20Z

Ah, good point @jbaiter -- I had forgotten that would be a roadblock.

For purposes like this, I've been meaning to replace the current implementations of TrackInfo and AlbumInfo with an "unstructured dict-like" alternative, where you could add arbitrary fields which will then be transferred into Item and Album objects when they're applied. Maybe this is the excuse we need to implement that style, which would then make this feature easy?

jbaiter · 2016-06-28T19:31:03Z

Sounds good to me, however I'm afraid my knowledge of beets internals is too limited to tackle that task.
If you provide the API, however, I'll gladly adapt the plugin :-)

sampsyo · 2016-06-28T23:15:29Z

It's a deal—I'll check back in here after I make progress on that refactoring.

itsbrex · 2016-08-09T23:52:29Z

@sampsyo just wondering if you have made any progress on this. Not a programmer myself, otherwise I'd take a stab at it. Apologize in advance if I sound snobby, not meaning to. 😄

sampsyo · 2016-08-10T03:21:34Z

No, sorry—I've been distracted with a few other things. I do hope to get to it soon.

itsbrex · 2016-08-10T06:50:53Z

@sampsyo Right on, no worries. :)

itsbrex · 2016-10-27T16:03:56Z

I've posted a bounty for this to get implemented if anyone is interested. I just really need this feature so figured I'd put up some cash to make it happen! 😄

https://www.bountysource.com/issues/35434835-beatport-fetch-musical-key-and-bpm-data

itsbrex · 2016-10-27T21:35:21Z

Btw, I've noticed the Genre tag is missing as well now on every track I import.

itsbrex · 2016-12-18T23:17:07Z

Anybody up for tackling this?
https://www.bountysource.com/issues/35434835-beatport-fetch-musical-key-and-bpm-data

exislow · 2017-04-29T13:11:59Z

I really need this, too! Is anybody up for it?

vifino · 2019-03-25T14:22:51Z

I too would like to see this. The BPM/Genre is quite important for me.
On Beatport, it's spot-on all the time.

wualex · 2019-09-13T17:58:54Z

Would also love to see this implemented and added some $ to the bounty :)

https://www.bountysource.com/issues/35434835-beatport-fetch-musical-key-bpm-and-genre-data

@BrianRoach you seem to have had the same issues as i do now (including the one with "original mix" & not finding some tracks), did you find any other solution after all?

temrix · 2019-09-14T13:57:00Z

I am currently tackling this issue.

I have already implemented BPM. Musical key should be equivalently easy. Beatport provides genre data for each track while beets has a genre per album. I am currently testing how this can be done best.

However, the API hooks need to be modified for this. Maybe I can modify it such that they take arbitrary values.

wualex · 2019-09-20T13:57:11Z

Silly question now... is there an "easier" way to update my beets to include this fix than to get the code from Github and install manually?
i have it on an Ubuntu 18.04 vm, installed by pip

temrix · 2019-09-20T14:15:38Z

pip can directly install the code from a git url: pip install git+https://github.com/beetbox/beets.git

This will clone the repo to a temporary directory and then install it via pip.

wualex · 2019-09-24T15:03:35Z

It seems that it still doesn't get the tags properly from beatport

Just for testing i tried the following tracks by using beet import -s /test/path (tried without -s also):
Alex Niggemann - Bwana (afro house)
Avidus - Psalm 23 (melodic house & techno)
Davi - The Gates Of Babylon (progressive house)
Stefano Ritteri - Mutanda (deep house)

...and selected beatport as source on every one but it still doesn't get the genre from it, at least

# metaflac --export-tags-to=- \$albumartist_The\ Gates\ of\ Babylon.flac | grep -i genre GENRE=Electro

The other 3 tracks are tagged as genre Dance
Am i doing something wrong or... ?

my config:

paths:
    default: $genre/$year_$month/$albumartist_$title
    singleton: $genre/$year_$month/$albumartist_$title
library: /mnt/d/Music/musiclibrary.db

replace:
    '[\\/]': _
    ^\.: _
    '[\x00-\x1f]': _
    '[<>:"\?\*\|]': _
    \.$: _
    \s+$: ''
original_date: yes
path_sep_replace: _

plugins: inline beatport
directory: /mnt/d/Music

import:
    move: no
    copy: yes
    write: yes
beatport:
    source_weight: 0.0
    tokenfile: beatport_token.json
    apikey: REDACTED
    apisecret: REDACTED
pathfields: {}
album_fields: {}
item_fields: {}

temrix · 2019-09-24T17:34:21Z

I guess, the data is coming from somewhere else, maybe discogs.

How did you even select the source? I have not found out myself how I can select to use Beatport other than doing `beet import ~/music/ --search-id [id].

wualex · 2019-09-24T19:18:06Z

It should take it from beatport, otherwise what's the point? :)

By "select the source" i was referring when beets asks me from where to correct the tags on import, you have different candidates options, and some of them would be from beatport, i would select that
But after i added source_weight to the config it mostly says it gets the data from beatport by default, otherwise i would manually pick beatport

But now that i look at the similarity line from when i import i see that it shows what tags it found and wants to correct, and from all my tests and options given by beets it only seems to want to correct artist, title, source, length - so it doesn't take into consideration genre at all, it just assumes the default (discogs?) is fine, when... it's not really 😬

@sampsyo could you shed some light on this please?

sampsyo · 2019-09-24T21:44:30Z

Beets does not use genre for matching. But it will apply new genres to your music, if they're available from the data source. Those won't show up in that list you're referring to, though, which only shows information about the matches.

Also, to see the data on your music, I recommend using something like beet ls -f '$title - $genre' or beet ls -af '$album - $genre'.

@wualex, when you report a problem, it's really useful to provide a complete set of steps so the developer can reproduce the problem. Maybe you can think carefully about how to help @temrix do exactly what you're doing to produce the same results.

wualex · 2019-09-25T08:52:58Z

At this point the only step and only thing i'm doing is beets import -s /mnt/d/test and even though beets says it corrects tags from beatport as you can see below:

/mnt/d/test/Alex Niggemann - Bwana (Original Mix).flac
Correcting track tags from:
    Alex Niggemann - Bwana (Original Mix) 
To:
    Alex Niggemann - Bwana
URL:
    https://beatport.com/track/bwana-original-mix/11381810
(Similarity: 93.8%) (title) (Beatport)
[A]pply, More candidates, Skip, Use as-is, Enter search, enter Id, aBort? A

/mnt/d/test/Avidus - Psalm 23.flac
Tagging track: Avidus - Psalm 23
URL:
    https://beatport.com/track/psalm-23-original-mix/11717455
(Similarity: 100.0%) (Beatport)

/mnt/d/test/Davi - The Gates Of Babylon (Original Mix).flac
Correcting track tags from:  
  Davi - The Gates Of Babylon (Original Mix)
To: 
   DAVI - The Gates Of Babylon
URL:
   https://beatport.com/track/the-gates-of-babylon-original-mix/6061936
(Similarity: 96.5%) (title) (Beatport)

/mnt/d/test/Kellerkind - Shakti Pan (feat. Sven).flac
Correcting track tags from:
   Kellerkind;Sven - Shakti Pan (feat. Sven)
To: 
   Kellerkind - Shakti Pan feat. Sven
URL:
   https://beatport.com/track/shakti-pan-feat-sven-original-mix/8712463
(Similarity: 94.3%) (artist) (Beatport)
[A]pply, More candidates, Skip, Use as-is, Enter search, enter Id, aBort? A

/mnt/d/test/Tunnelvisions - Guava (Extended Mix).flac
Tagging track: Tunnelvisions - Guava (Extended Mix)
URL: 
   https://beatport.com/track/guava-extended-mix/10227696
(Similarity: 100.0%) (Beatport)

The result is as you can see wrong:

beet ls -f '$artist - $title - $genre'

Alex Niggemann - Bwana - Electro
Avidus - Psalm 23 - Electro
DAVI - The Gates Of Babylon - Electro
Kellerkind - Shakti Pan feat. Sven - Electro
Tunnelvisions - Guava (Extended Mix) - Dance

And now i don't know what else i can provide but i'm up for more testing or if you have questions....
Thanks.

flovanc · 2019-09-26T13:19:24Z

maybe there is a mistake between "genre" and "subgenre" from beatport.
what we need is "subgenre" from beatport copied inside genre id3 from audiofile ...
edit:
after investigation on beatport api i'm wrong ...
i wait for news

temrix · 2019-09-26T15:01:37Z

I have finally reproduced it and I will file a new issue because this here is already closed. In the new issue I will explain more thoroughly what the problem actually is. Just give me some time.

sampsyo added the feature features we would like to implement label Jun 23, 2016

sampsyo changed the title ~~Beatport: Key, BPM & sometimes Genre tags missing~~ beatport: Fetch musical key and BPM data Jun 23, 2016

sampsyo mentioned this issue Jun 29, 2016

Clarify remote_priority? #2092

Closed

sampsyo changed the title ~~beatport: Fetch musical key and BPM data~~ beatport: Fetch musical key, BPM, and genre data Apr 21, 2019

sampsyo mentioned this issue Apr 21, 2019

Beatport Genres #3229

Closed

temrix mentioned this issue Sep 18, 2019

Fix #2080 #3372

Merged

sampsyo closed this as completed in db7ef0e Sep 19, 2019

temrix mentioned this issue Oct 1, 2019

Beatport plugin does not apply tags to file #3387

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

beatport: Fetch musical key, BPM, and genre data #2080

beatport: Fetch musical key, BPM, and genre data #2080

itsbrex commented Jun 23, 2016 •

edited

Loading

sampsyo commented Jun 23, 2016

itsbrex commented Jun 23, 2016

jbaiter commented Jun 28, 2016 •

edited

Loading

sampsyo commented Jun 28, 2016

jbaiter commented Jun 28, 2016

sampsyo commented Jun 28, 2016

itsbrex commented Aug 9, 2016

sampsyo commented Aug 10, 2016

itsbrex commented Aug 10, 2016

itsbrex commented Oct 27, 2016

itsbrex commented Oct 27, 2016

itsbrex commented Dec 18, 2016

exislow commented Apr 29, 2017

vifino commented Mar 25, 2019

wualex commented Sep 13, 2019 •

edited

Loading

temrix commented Sep 14, 2019

wualex commented Sep 20, 2019

temrix commented Sep 20, 2019

wualex commented Sep 24, 2019 •

edited

Loading

temrix commented Sep 24, 2019

wualex commented Sep 24, 2019 •

edited

Loading

sampsyo commented Sep 24, 2019

wualex commented Sep 25, 2019 •

edited

Loading

flovanc commented Sep 26, 2019 •

edited

Loading

temrix commented Sep 26, 2019

beatport: Fetch musical key, BPM, and genre data #2080

beatport: Fetch musical key, BPM, and genre data #2080

Comments

itsbrex commented Jun 23, 2016 • edited Loading

Problem

Setup

sampsyo commented Jun 23, 2016

itsbrex commented Jun 23, 2016

jbaiter commented Jun 28, 2016 • edited Loading

sampsyo commented Jun 28, 2016

jbaiter commented Jun 28, 2016

sampsyo commented Jun 28, 2016

itsbrex commented Aug 9, 2016

sampsyo commented Aug 10, 2016

itsbrex commented Aug 10, 2016

itsbrex commented Oct 27, 2016

itsbrex commented Oct 27, 2016

itsbrex commented Dec 18, 2016

exislow commented Apr 29, 2017

vifino commented Mar 25, 2019

wualex commented Sep 13, 2019 • edited Loading

temrix commented Sep 14, 2019

wualex commented Sep 20, 2019

temrix commented Sep 20, 2019

wualex commented Sep 24, 2019 • edited Loading

temrix commented Sep 24, 2019

wualex commented Sep 24, 2019 • edited Loading

sampsyo commented Sep 24, 2019

wualex commented Sep 25, 2019 • edited Loading

flovanc commented Sep 26, 2019 • edited Loading

temrix commented Sep 26, 2019

itsbrex commented Jun 23, 2016 •

edited

Loading

jbaiter commented Jun 28, 2016 •

edited

Loading

wualex commented Sep 13, 2019 •

edited

Loading

wualex commented Sep 24, 2019 •

edited

Loading

wualex commented Sep 24, 2019 •

edited

Loading

wualex commented Sep 25, 2019 •

edited

Loading

flovanc commented Sep 26, 2019 •

edited

Loading