Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Crash when copying newly imported files #2537

Closed
jansol opened this issue May 1, 2017 · 18 comments
Closed

Crash when copying newly imported files #2537

jansol opened this issue May 1, 2017 · 18 comments
Labels
needinfo We need more details or follow-up from the filer before this can be tagged "bug" or "feature."

Comments

@jansol
Copy link

jansol commented May 1, 2017

Problem

Downloaded https://ultimae.bandcamp.com/album/hanging-masses and tried to import it, causing beets to crash between updating the db and copying the files to the media dir.

Running this command in verbose (-vv) mode:

$ beet -vv import -t CELL\ -\ -\ Hanging\ Masses

Led to this problem:

Traceback (most recent call last):
  File "/nix/store/0k3smkrz6wwn9zmk2l5l49g1k1ngi85z-beets-1.4.3/bin/.beet-wrapped", line 12, in <module>
    sys.exit(main())
  File "/nix/store/0k3smkrz6wwn9zmk2l5l49g1k1ngi85z-beets-1.4.3/lib/python2.7/site-packages/beets/ui/__init__.py", line 1209, in main
    _raw_main(args)
  File "/nix/store/0k3smkrz6wwn9zmk2l5l49g1k1ngi85z-beets-1.4.3/lib/python2.7/site-packages/beets/ui/__init__.py", line 1196, in _raw_main
    subcommand.func(lib, suboptions, subargs)
  File "/nix/store/0k3smkrz6wwn9zmk2l5l49g1k1ngi85z-beets-1.4.3/lib/python2.7/site-packages/beets/ui/commands.py", line 930, in import_func
    import_files(lib, paths, query)
  File "/nix/store/0k3smkrz6wwn9zmk2l5l49g1k1ngi85z-beets-1.4.3/lib/python2.7/site-packages/beets/ui/commands.py", line 907, in import_files
    session.run()
  File "/nix/store/0k3smkrz6wwn9zmk2l5l49g1k1ngi85z-beets-1.4.3/lib/python2.7/site-packages/beets/importer.py", line 319, in run
    pl.run_parallel(QUEUE_SIZE)
  File "/nix/store/0k3smkrz6wwn9zmk2l5l49g1k1ngi85z-beets-1.4.3/lib/python2.7/site-packages/beets/util/pipeline.py", line 445, in run_parallel
    six.reraise(exc_info[0], exc_info[1], exc_info[2])
  File "/nix/store/0k3smkrz6wwn9zmk2l5l49g1k1ngi85z-beets-1.4.3/lib/python2.7/site-packages/beets/util/pipeline.py", line 312, in run
    out = self.coro.send(msg)
  File "/nix/store/0k3smkrz6wwn9zmk2l5l49g1k1ngi85z-beets-1.4.3/lib/python2.7/site-packages/beets/util/pipeline.py", line 194, in coro
    func(*(args + (task,)))
  File "/nix/store/0k3smkrz6wwn9zmk2l5l49g1k1ngi85z-beets-1.4.3/lib/python2.7/site-packages/beets/importer.py", line 1392, in plugin_stage
    func(session, task)
  File "/nix/store/0k3smkrz6wwn9zmk2l5l49g1k1ngi85z-beets-1.4.3/lib/python2.7/site-packages/beets/plugins.py", line 124, in wrapper
    return func(*args, **kwargs)
  File "/nix/store/0k3smkrz6wwn9zmk2l5l49g1k1ngi85z-beets-1.4.3/lib/python2.7/site-packages/beetsplug/keyfinder.py", line 51, in imported
    self.find_key(task.items)
  File "/nix/store/0k3smkrz6wwn9zmk2l5l49g1k1ngi85z-beets-1.4.3/lib/python2.7/site-packages/beetsplug/keyfinder.py", line 86, in find_key
    item.store()
  File "/nix/store/0k3smkrz6wwn9zmk2l5l49g1k1ngi85z-beets-1.4.3/lib/python2.7/site-packages/beets/library.py", line 326, in store
    super(LibModel, self).store(fields)
  File "/nix/store/0k3smkrz6wwn9zmk2l5l49g1k1ngi85z-beets-1.4.3/lib/python2.7/site-packages/beets/dbcore/db.py", line 352, in store
    self._check_db()
  File "/nix/store/0k3smkrz6wwn9zmk2l5l49g1k1ngi85z-beets-1.4.3/lib/python2.7/site-packages/beets/dbcore/db.py", line 206, in _check_db
    u'{0} has no database'.format(type(self).__name__)
ValueError: Item has no database

Setup

  • OS: NixOS
  • Python version: 2.7.13
  • beets version: 1.4.3
  • Turning off plugins made problem go away (yes/no):

My configuration (output of beet config) is:

ignore_hidden: yes

paths:
    default: '%the{$albumartist}/%the{$album}/$track-$title'
    singleton: '%the{$artist}-%the{$title}'
    comp: '%the{$album}/$track-$title'
    albumtype:soundtrack: Soundtracks/%the{$album}/$track-$title
    albumtype:single: '%the{$albumartist}/%the{$album}/$track-$title'
convert:
    embed: yes
    album_art_maxwidth: 900
    format: aac
    dest: /nas/media/library_aac
    copy_album_art: yes
    formats:
        aac:
            command: ffmpeg -i $source -y -vn -acodec aac -aq 1 -ar 48k $dest
            extension: m4a
        opus:
            command: ffmpeg -i $source -y -vn -acodec libopus -ab 128k $dest
        version:
            command: ffmpeg
        alac:
            command: ffmpeg -i $source -y -vn -acodec alac $dest
            extension: m4a
        mp3: ffmpeg -i $source -y -vn -aq 2 $dest
        flac: ffmpeg -i $source -y -vn -acodec flac $dest
        ogg: ffmpeg -i $source -y -vn -acodec libvorbis -aq 3 $dest
        wma: ffmpeg -i $source -y -vn -acodec wmav2 -vn $dest
    never_convert_lossy_files: no
    auto: no
    threads: 4
    tmpdir:
    max_bitrate: 500

    paths: {}
    pretend: no
    quiet: no
threaded: yes
art_filename: folder
fetchart:
    sources: [filesystem]
    store_source: yes
    auto: yes
    minwidth: 0
    google_engine: 001442825323518660753:hrh5ch1gjzm
    enforce_ratio: no
    cautious: no
    maxwidth: 0
    google_key: REDACTED
    fanarttv_key: REDACTED
    cover_names:
    - cover
    - front
    - art
    - album
    - folder
play:
    command: mpc insert
    relative_to: /media/music
    warning_treshold: 30
    raw: yes
    use_folders: no
    warning_threshold: 100
fuzzy:
    prefix: "\xA7"
    threshold: 0.7

replace:
    '[\\/]': _
    ^\.: _
    '[\x00-\x1f]': _
    '[<>:"\?\*\|]': _
    \.$: _
    \s+$: ''
    ^\s+: ''
ignore: .AppleDouble ._* *~ .DS_Store
replaygain:
    backend: bs1770gain
    method: ebu
    targetlevel: 89
    auto: yes
    overwrite: no
    chunk_at: 5000

ui:
    color: yes

plugins: acousticbrainz badfiles convert edit embedart fetchart ftintitle fuzzy info keyfinder mbsync missing play replaygain random the web
directory: /media/music

import:
    incremental: yes
the:
    a: yes
    patterns: []
    the: yes
    strip: no
    format: '{0}, {1}'
keyfinder:
    bin: keyfinder-cli
    auto: yes
    overwrite: no
embedart:
    compare_threshold: 0
    auto: yes
    ifempty: no
    remove_art_file: no
    maxwidth: 0
web:
    host: 127.0.0.1
    port: 8337
    cors: ''
missing:
    count: no
    total: no
acousticbrainz:
    auto: yes
    force: no
ftintitle:
    auto: yes
    drop: no
    format: feat. {0}
edit:
    itemfields: track title artist album
    albumfields: album albumartist
    ignore_fields: id path
@sampsyo sampsyo added the needinfo We need more details or follow-up from the filer before this can be tagged "bug" or "feature." label May 1, 2017
@sampsyo
Copy link
Member

sampsyo commented May 1, 2017

That's certainly strange! It looks like this is coming from the keyfinder plugin. Can you try running that plugin on other music too to check whether this is specific to certain albums? Verbose output from a run where this worked and one where it didn't could be really helpful.

@jansol
Copy link
Author

jansol commented May 1, 2017

I'll try tomorrow. I suspect it's related to the last track being a continuous mix of the whole album (i.e. very long) and in FLAC format.

@jansol
Copy link
Author

jansol commented May 2, 2017

Curiously this only happened during import, disabling the automatic keyfinder scan on import and running the plugin on the problematic file manually worked perfectly fine.

@sampsyo
Copy link
Member

sampsyo commented May 2, 2017

Strange! I'm pretty stumped about how this particular operation could fail during import… do you know whether this happens every time if you try importing the problematic file again?

@jansol
Copy link
Author

jansol commented May 2, 2017

Yup, happened everytime I tried. No idea if related, but this particular file is not shown in the MusicBrainz entry (most likely that one does not refer to the bandcamp release).

@sampsyo
Copy link
Member

sampsyo commented May 2, 2017

OK; thanks! So I can try to reproduce the problem, do you know where I can get a copy of this file (or another offending track)?

@jansol
Copy link
Author

jansol commented May 2, 2017

There's a bandcamp link in the OP. Or you can poke me on IRC.

@sampsyo
Copy link
Member

sampsyo commented May 5, 2017

OK, cool—I might try to find you on IRC for a copy of the file. In the mean time, though, I should have asked: is there anything useful in the verbose log leading up to the crash?

@jansol
Copy link
Author

jansol commented Jun 15, 2017

No, up to the backtrace everything is going peachy. (just retried with an empty db)

Full log:

% beet -vvv import -t ~/Downloads/CELL\ -\ -\ Hanging\ Masses
user configuration: /home/xxx/.config/beets/config.yaml
data directory: /home/xxx/.config/beets
plugin paths: 
Sending event: pluginload
library database: /home/xxx/.config/beets/library.db
library directory: /home/xxx/tmp/beetstest
Sending event: library_opened
Sending event: import_begin
state file could not be read: [Errno 2] No such file or directory: u'/home/xxx/.config/beets/state.pickle'
state file could not be read: [Errno 2] No such file or directory: u'/home/xxx/.config/beets/state.pickle'
Sending event: import_task_created
Sending event: import_task_start
Looking up: /home/xxx/Downloads/CELL - - Hanging Masses
Tagging CELL - [ Hanging Masses ]
Searching for discovered album ID: d62769e7-12be-41f6-9561-ca822f0b336c
Candidate: Cell - Hanging Masses
Success. Distance: 0.08
Album ID match recommendation is Recommendation.medium
Search terms: CELL - [ Hanging Masses ]
Album might be VA: True
Sending event: albuminfo_received
Candidate: Cell - Hanging Masses
Success. Distance: 0.08
Sending event: albuminfo_received
Candidate: Cell - Hanging Masses
Duplicate.
Sending event: albuminfo_received
Candidate: Cell - Phonic Peace
Success. Distance: 0.60
Sending event: albuminfo_received
Candidate: Cell Division - Dissolve
Success. Distance: 0.75
Sending event: albuminfo_received
Candidate: Soft Cell - The Twelve Inch Singles Collection (disc 1)
Success. Distance: 0.71
Sending event: albuminfo_received
Candidate: Various Artists - 東方音伽盤[改]
Success. Distance: 0.73
Sending event: albuminfo_received
Candidate: Various Artists - 연가 [戀歌]
Success. Distance: 0.94
Sending event: albuminfo_received
Candidate: Clausthome - Radiophare [ Lampe d'Ambiance ]
Success. Distance: 0.75
Sending event: albuminfo_received
Candidate: Various Artists - RGB [ An Audio Spectrum ]
Success. Distance: 0.86
Sending event: albuminfo_received
Candidate: Various Artists - ベルサイユのばら 音楽集 [完全版]
Success. Distance: 0.95
Evaluating 10 candidates.

/home/xxx/Downloads/CELL - - Hanging Masses (9 items)
Sending event: before_choose_candidate
Correcting tags from:
    CELL - [ Hanging Masses ]
To:
    Cell - Hanging Masses
URL:
    https://musicbrainz.org/release/b3fb436b-b1f0-4fb9-9651-92bf00873848
(Similarity: 92.4%) (tracks, unmatched tracks) (CD, 2009, FR, Ultimae Records)
 * Calling (9:03)                   -> Calling (8:52) (length)
 * Switch Off (8:46)                -> Switch Off (8:30) (length)
 * Second Shape part I (9:19)       -> Second Shape, Part I (9:02) (length)
 * Risky Nap Under Blue Tree (6:59) -> Risky Nap Under Blue Tree (6:35) (length)
 * Hanging Masses (11:02)           -> Hanging Masses (10:48) (length)
 * Vapor (8:49)                     -> Vapor (8:32) (length)
 * Second Shape part II (9:27)      -> Second Shape, Part II (8:59) (length)
Unmatched tracks (1):
 ! [ Hanging Masses ] | Full Mix (#9) (72:36)
[A]pply, More candidates, Skip, Use as-is, as Tracks, Group albums,
Enter search, enter Id, aBort, eDit, edit Candidates? m
Finding tags for album "CELL - [ Hanging Masses ]".
Candidates:
1. Cell - Hanging Masses (92.4%) (tracks, unmatched tracks) (CD, 2009, FR, Ultimae Records)
2. Cell - Hanging Masses (92.4%) (tracks, unmatched tracks) (Digital Media, 2009, XW, Ultimae Records)
3. Cell - Phonic Peace (39.8%) (tracks, album, year) (CD, 2005, IT, Indica Music)
4. Soft Cell - The Twelve Inch Singles Collection (disc 1) (28.6%) (tracks, album, artist)
5. Various Artists - 東方音伽盤[改] (27.3%) (tracks, album, year) (CD, 2008, JP, Black Label Records)
6. Cell Division - Dissolve (25.3%) (tracks, album, artist, ...) (CD, 2002, CH)
7. Clausthome - Radiophare [ Lampe d'Ambiance ] (24.6%) (tracks, artist, album, ...) (CD, 2005, RU, Indiestate Distribution)
8. Various Artists - RGB [ An Audio Spectrum ] (14.2%) (tracks, unmatched tracks, album, ...) (CD, 1998, GB, Source Research)
9. Various Artists - 연가 [戀歌] (6.1%) (missing tracks, tracks, album, ...) (4xCD, 2001, KR, DOREMI)
10. Various Artists - ベルサイユのばら 音楽集 [完全版] (5.2%) (missing tracks, tracks, album, ...) (3xCD, 2016, JP, USM Japan, 初回限定盤)
# selection (default 1), Skip, Use as-is, as Tracks, Group albums,
Enter search, enter Id, aBort, eDit, edit Candidates? 2
Correcting tags from:
    CELL - [ Hanging Masses ]
To:
    Cell - Hanging Masses
URL:
    https://musicbrainz.org/release/d62769e7-12be-41f6-9561-ca822f0b336c
(Similarity: 92.4%) (tracks, unmatched tracks) (Digital Media, 2009, XW, Ultimae Records)
 * Calling (9:03)                   -> Calling (8:52) (length)
 * Switch Off (8:46)                -> Switch Off (8:30) (length)
 * Second Shape part I (9:19)       -> Second Shape, Part I (9:02) (length)
 * Risky Nap Under Blue Tree (6:59) -> Risky Nap Under Blue Tree (6:35) (length)
 * Hanging Masses (11:02)           -> Hanging Masses (10:48) (length)
 * Vapor (8:49)                     -> Vapor (8:32) (length)
 * Second Shape part II (9:27)      -> Second Shape, Part II (8:59) (length)
Unmatched tracks (1):
 ! [ Hanging Masses ] | Full Mix (#9) (72:36)
Apply, More candidates, Skip, Use as-is, as Tracks, Group albums,
Enter search, enter Id, aBort, eDit, edit Candidates? a
Sending event: import_task_choice
Sending event: import_task_apply
0 of 8 items replaced
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
fetchart: trying source filesystem for album Cell - Hanging Masses
fetchart: using well-named art file cover.jpg
fetchart: using local image /home/xxx/Downloads/CELL - - Hanging Masses/cover.jpg
replaygain: analyzing Cell - Hanging Masses
replaygain: executing /nix/store/j4fsin9p67r7kwk22k6haa8cs2yz95ic-bs1770gain-0.4.12/bin/bs1770gain --ebu -p /home/xxx/Downloads/CELL - - Hanging Masses/CELL - - Hanging Masses - - 01 Calling.flac /home/xxx/Downloads/CELL - - Hanging Masses/CELL - - Hanging Masses - - 03 Second Shape part I.flac /home/xxx/Downloads/CELL - - Hanging Masses/CELL - - Hanging Masses - - 04 Risky Nap Under Blue Tree.flac /home/xxx/Downloads/CELL - - Hanging Masses/CELL - - Hanging Masses - - 05 Hanging Masses.flac /home/xxx/Downloads/CELL - - Hanging Masses/CELL - - Hanging Masses - - 06 Vapor.flac /home/xxx/Downloads/CELL - - Hanging Masses/CELL - - Hanging Masses - - 07 Second Shape part II.flac /home/xxx/Downloads/CELL - - Hanging Masses/CELL - - Hanging Masses - - 08 Universal Sunrise.flac /home/xxx/Downloads/CELL - - Hanging Masses/CELL - - Hanging Masses - - 02 Switch Off.flac
replaygain: analysis finished: analyzing ...
  [1/8] "CELL - - Hanging Masses - - 01 Calling.flac":         
       integrated:  -19.05 LUFS / -3.95 LU
      sample peak:  -1.10 SPFS / 0.881344
  [2/8] "CELL - - Hanging Masses - - 02 Switch Off.flac":         
       integrated:  -13.99 LUFS / -9.01 LU
      sample peak:  -0.70 SPFS / 0.922758
  [3/8] "CELL - - Hanging Masses - - 03 Second Shape part I.flac":         
       integrated:  -14.17 LUFS / -8.83 LU
      sample peak:  -0.74 SPFS / 0.917905
  [4/8] "CELL - - Hanging Masses - - 04 Risky Nap Under Blue Tree.flac":         
       integrated:  -13.38 LUFS / -9.62 LU
      sample peak:  -0.70 SPFS / 0.922788
  [5/8] "CELL - - Hanging Masses - - 05 Hanging Masses.flac":         
       integrated:  -13.17 LUFS / -9.83 LU
      sample peak:  -0.70 SPFS / 0.922605
  [6/8] "CELL - - Hanging Masses - - 06 Vapor.flac":         
       integrated:  -18.18 LUFS / -4.82 LU
      sample peak:  -0.90 SPFS / 0.901456
  [7/8] "CELL - - Hanging Masses - - 07 Second Shape part II.flac":         
       integrated:  -15.23 LUFS / -7.77 LU
      sample peak:  -0.85 SPFS / 0.906827
  [8/8] "CELL - - Hanging Masses - - 08 Universal Sunrise.flac":         
       integrated:  -19.13 LUFS / -3.87 LU
      sample peak:  -1.04 SPFS / 0.886654
  [ALBUM]:
       integrated:  -15.33 LUFS / -7.67 LU
      sample peak:  -0.70 SPFS / 0.922788
done.

replaygain: 8 items, 9 results
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
replaygain: applied album gain -7.67, peak 0.922788
Sending event: database_change
replaygain: applied track gain -3.95, peak 0.881344
Sending event: database_change
replaygain: applied track gain -9.01, peak 0.922758
Sending event: database_change
replaygain: applied track gain -8.83, peak 0.917905
Sending event: database_change
replaygain: applied track gain -9.62, peak 0.922788
Sending event: database_change
replaygain: applied track gain -9.83, peak 0.922605
Sending event: database_change
replaygain: applied track gain -4.82, peak 0.901456
Sending event: database_change
replaygain: applied track gain -7.77, peak 0.906827
Sending event: database_change
replaygain: applied track gain -3.87, peak 0.886654
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
ftintitle: /home/xxx/Downloads/CELL - - Hanging Masses/CELL - - Hanging Masses - - 02 Switch Off.flac
ftintitle: artist: Cell feat. Aes Dana -> Cell
ftintitle: title: Switch Off -> Switch Off feat. Aes Dana
Sending event: database_change
keyfinder: added computed initial key Am for /home/xxx/Downloads/CELL - - Hanging Masses/CELL - - Hanging Masses - - 01 Calling.flac
Sending event: database_change
keyfinder: added computed initial key Am for /home/xxx/Downloads/CELL - - Hanging Masses/CELL - - Hanging Masses - - 02 Switch Off.flac
Sending event: database_change
keyfinder: added computed initial key Db for /home/xxx/Downloads/CELL - - Hanging Masses/CELL - - Hanging Masses - - 03 Second Shape part I.flac
Sending event: database_change
keyfinder: added computed initial key Ebm for /home/xxx/Downloads/CELL - - Hanging Masses/CELL - - Hanging Masses - - 04 Risky Nap Under Blue Tree.flac
Sending event: database_change
keyfinder: added computed initial key Bbm for /home/xxx/Downloads/CELL - - Hanging Masses/CELL - - Hanging Masses - - 05 Hanging Masses.flac
Sending event: database_change
keyfinder: added computed initial key Bbm for /home/xxx/Downloads/CELL - - Hanging Masses/CELL - - Hanging Masses - - 07 Second Shape part II.flac
Sending event: database_change
keyfinder: added computed initial key Ab for /home/xxx/Downloads/CELL - - Hanging Masses/CELL - - Hanging Masses - - 08 Universal Sunrise.flac
Sending event: database_change
keyfinder: added computed initial key Bbm for /home/xxx/Downloads/CELL - - Hanging Masses/CELL - - Hanging Masses - - 09 - Hanging Masses - - Full Mix.flac
Traceback (most recent call last):
  File "/nix/store/7x2zx4d8mf8qfsxgs8ygbl9ba6df5gyi-beets-1.4.3/bin/.beet-wrapped", line 12, in <module>
    sys.exit(main())
  File "/nix/store/7x2zx4d8mf8qfsxgs8ygbl9ba6df5gyi-beets-1.4.3/lib/python2.7/site-packages/beets/ui/__init__.py", line 1209, in main
    _raw_main(args)
  File "/nix/store/7x2zx4d8mf8qfsxgs8ygbl9ba6df5gyi-beets-1.4.3/lib/python2.7/site-packages/beets/ui/__init__.py", line 1196, in _raw_main
    subcommand.func(lib, suboptions, subargs)
  File "/nix/store/7x2zx4d8mf8qfsxgs8ygbl9ba6df5gyi-beets-1.4.3/lib/python2.7/site-packages/beets/ui/commands.py", line 930, in import_func
    import_files(lib, paths, query)
  File "/nix/store/7x2zx4d8mf8qfsxgs8ygbl9ba6df5gyi-beets-1.4.3/lib/python2.7/site-packages/beets/ui/commands.py", line 907, in import_files
    session.run()
  File "/nix/store/7x2zx4d8mf8qfsxgs8ygbl9ba6df5gyi-beets-1.4.3/lib/python2.7/site-packages/beets/importer.py", line 319, in run
    pl.run_parallel(QUEUE_SIZE)
  File "/nix/store/7x2zx4d8mf8qfsxgs8ygbl9ba6df5gyi-beets-1.4.3/lib/python2.7/site-packages/beets/util/pipeline.py", line 445, in run_parallel
    six.reraise(exc_info[0], exc_info[1], exc_info[2])
  File "/nix/store/7x2zx4d8mf8qfsxgs8ygbl9ba6df5gyi-beets-1.4.3/lib/python2.7/site-packages/beets/util/pipeline.py", line 312, in run
    out = self.coro.send(msg)
  File "/nix/store/7x2zx4d8mf8qfsxgs8ygbl9ba6df5gyi-beets-1.4.3/lib/python2.7/site-packages/beets/util/pipeline.py", line 194, in coro
    func(*(args + (task,)))
  File "/nix/store/7x2zx4d8mf8qfsxgs8ygbl9ba6df5gyi-beets-1.4.3/lib/python2.7/site-packages/beets/importer.py", line 1392, in plugin_stage
    func(session, task)
  File "/nix/store/7x2zx4d8mf8qfsxgs8ygbl9ba6df5gyi-beets-1.4.3/lib/python2.7/site-packages/beets/plugins.py", line 124, in wrapper
    return func(*args, **kwargs)
  File "/nix/store/7x2zx4d8mf8qfsxgs8ygbl9ba6df5gyi-beets-1.4.3/lib/python2.7/site-packages/beetsplug/keyfinder.py", line 51, in imported
    self.find_key(task.items)
  File "/nix/store/7x2zx4d8mf8qfsxgs8ygbl9ba6df5gyi-beets-1.4.3/lib/python2.7/site-packages/beetsplug/keyfinder.py", line 86, in find_key
    item.store()
  File "/nix/store/7x2zx4d8mf8qfsxgs8ygbl9ba6df5gyi-beets-1.4.3/lib/python2.7/site-packages/beets/library.py", line 326, in store
    super(LibModel, self).store(fields)
  File "/nix/store/7x2zx4d8mf8qfsxgs8ygbl9ba6df5gyi-beets-1.4.3/lib/python2.7/site-packages/beets/dbcore/db.py", line 352, in store
    self._check_db()
  File "/nix/store/7x2zx4d8mf8qfsxgs8ygbl9ba6df5gyi-beets-1.4.3/lib/python2.7/site-packages/beets/dbcore/db.py", line 206, in _check_db
    u'{0} has no database'.format(type(self).__name__)
ValueError: Item has no database

@sampsyo
Copy link
Member

sampsyo commented Jun 15, 2017

Hmm… I'm suddenly suspicious that this might have to do with interference with another plugin. Is there any chance you can try temporarily disabling all your plugins except for keyfinder to see if that narrows anything down?

@jansol
Copy link
Author

jansol commented Mar 20, 2018

Same result with only keyfinder enabled: (log lines omitted as they are the same as before)

Traceback (most recent call last):
  File "/nix/store/q3ivzd6zflz4mrr87rpfb4bs6hh1maap-beets-1.4.6/bin/.beet-wrapped", line 12, in <module>
    sys.exit(main())
  File "/nix/store/q3ivzd6zflz4mrr87rpfb4bs6hh1maap-beets-1.4.6/lib/python2.7/site-packages/beets/ui/__init__.py", line 1256, in main
    _raw_main(args)
  File "/nix/store/q3ivzd6zflz4mrr87rpfb4bs6hh1maap-beets-1.4.6/lib/python2.7/site-packages/beets/ui/__init__.py", line 1243, in _raw_main
    subcommand.func(lib, suboptions, subargs)
  File "/nix/store/q3ivzd6zflz4mrr87rpfb4bs6hh1maap-beets-1.4.6/lib/python2.7/site-packages/beets/ui/commands.py", line 937, in import_func
    import_files(lib, paths, query)
  File "/nix/store/q3ivzd6zflz4mrr87rpfb4bs6hh1maap-beets-1.4.6/lib/python2.7/site-packages/beets/ui/commands.py", line 914, in import_files
    session.run()
  File "/nix/store/q3ivzd6zflz4mrr87rpfb4bs6hh1maap-beets-1.4.6/lib/python2.7/site-packages/beets/importer.py", line 327, in run
    pl.run_parallel(QUEUE_SIZE)
  File "/nix/store/q3ivzd6zflz4mrr87rpfb4bs6hh1maap-beets-1.4.6/lib/python2.7/site-packages/beets/util/pipeline.py", line 445, in run_parallel
    six.reraise(exc_info[0], exc_info[1], exc_info[2])
  File "/nix/store/q3ivzd6zflz4mrr87rpfb4bs6hh1maap-beets-1.4.6/lib/python2.7/site-packages/beets/util/pipeline.py", line 312, in run
    out = self.coro.send(msg)
  File "/nix/store/q3ivzd6zflz4mrr87rpfb4bs6hh1maap-beets-1.4.6/lib/python2.7/site-packages/beets/util/pipeline.py", line 194, in coro
    func(*(args + (task,)))
  File "/nix/store/q3ivzd6zflz4mrr87rpfb4bs6hh1maap-beets-1.4.6/lib/python2.7/site-packages/beets/importer.py", line 1506, in plugin_stage
    func(session, task)
  File "/nix/store/q3ivzd6zflz4mrr87rpfb4bs6hh1maap-beets-1.4.6/lib/python2.7/site-packages/beets/plugins.py", line 124, in wrapper
    return func(*args, **kwargs)
  File "/nix/store/q3ivzd6zflz4mrr87rpfb4bs6hh1maap-beets-1.4.6/lib/python2.7/site-packages/beetsplug/keyfinder.py", line 51, in imported
    self.find_key(task.items)
  File "/nix/store/q3ivzd6zflz4mrr87rpfb4bs6hh1maap-beets-1.4.6/lib/python2.7/site-packages/beetsplug/keyfinder.py", line 86, in find_key
    item.store()
  File "/nix/store/q3ivzd6zflz4mrr87rpfb4bs6hh1maap-beets-1.4.6/lib/python2.7/site-packages/beets/library.py", line 346, in store
    super(LibModel, self).store(fields)
  File "/nix/store/q3ivzd6zflz4mrr87rpfb4bs6hh1maap-beets-1.4.6/lib/python2.7/site-packages/beets/dbcore/db.py", line 385, in store
    self._check_db()
  File "/nix/store/q3ivzd6zflz4mrr87rpfb4bs6hh1maap-beets-1.4.6/lib/python2.7/site-packages/beets/dbcore/db.py", line 215, in _check_db
    u'{0} has no database'.format(type(self).__name__)
ValueError: Item has no database

@sampsyo
Copy link
Member

sampsyo commented Mar 20, 2018

Very troubling! It does seem like this must be coming from keyfinder, right? Does it ever happen if that plugin is disabled?

@jansol
Copy link
Author

jansol commented Mar 20, 2018

See earlier in the discussion - everything is fine if keyfinder is disabled on import. (even when running it manually afterwards)

@Hamuko
Copy link

Hamuko commented Apr 25, 2018 •
edited
Loading

I discovered this issue through Google by looking for "beets Item has no database". I'm still assuming that this is relevant, because I am having the same issue. However, it is with a different album, different plugin, different operating system, different beets version and even different Python architecture.

I have a plugin that adds albums imported into beets to the bottom of my MPD queue, so I can then listen to them later: beets-mpdqueue. It's fairly simple and I've written it myself. Used it with hundreds of imports, has worked well for >99% of the time. Sometimes it failed to connect to the MPD server, logging warning messages. Nothing too serious, nothing crashy.

However, when I tried importing Solanin by Mere (no minimum payment on Bandcamp), beets crashed and my plugin is actually part of the traceback.

Traceback (most recent call last):
  File "/usr/local/bin/beet", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/site-packages/beets/ui/__init__.py", line 1256, in main
    _raw_main(args)
  File "/usr/local/lib/python3.6/site-packages/beets/ui/__init__.py", line 1243, in _raw_main
    subcommand.func(lib, suboptions, subargs)
  File "/usr/local/lib/python3.6/site-packages/beets/ui/commands.py", line 937, in import_func
    import_files(lib, paths, query)
  File "/usr/local/lib/python3.6/site-packages/beets/ui/commands.py", line 914, in import_files
    session.run()
  File "/usr/local/lib/python3.6/site-packages/beets/importer.py", line 327, in run
    pl.run_parallel(QUEUE_SIZE)
  File "/usr/local/lib/python3.6/site-packages/beets/util/pipeline.py", line 445, in run_parallel
    six.reraise(exc_info[0], exc_info[1], exc_info[2])
  File "/usr/local/lib/python3.6/site-packages/six.py", line 686, in reraise
    raise value
  File "/usr/local/lib/python3.6/site-packages/beets/util/pipeline.py", line 358, in run
    self.coro.send(msg)
  File "/usr/local/lib/python3.6/site-packages/beets/util/pipeline.py", line 171, in coro
    task = func(*(args + (task,)))
  File "/usr/local/lib/python3.6/site-packages/beets/importer.py", line 1538, in manipulate_files
    session=session,
  File "/usr/local/lib/python3.6/site-packages/beets/importer.py", line 742, in manipulate_files
    plugins.send('import_task_files', session=session, task=self)
  File "/usr/local/lib/python3.6/site-packages/beets/plugins.py", line 452, in send
    result = handler(**arguments)
  File "/usr/local/lib/python3.6/site-packages/beets/plugins.py", line 124, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/beetsplug/mpdqueue.py", line 137, in import_task_files
    destination = item.destination(fragment=True)
  File "/usr/local/lib/python3.6/site-packages/beets/library.py", line 825, in destination
    self._check_db()
  File "/usr/local/lib/python3.6/site-packages/beets/dbcore/db.py", line 215, in _check_db
    u'{0} has no database'.format(type(self).__name__)

It calls a different part of the API, but it goes through the same self._check_db() method as with the original report and then crashes.

I'm on macOS 10.13.2. It's not a Mac though, just some PC hardware with macOS on it. Shouldn't be relevant here. I'm running beets version 1.4.6 on Python version 3.6.5.

Now, onto the interesting bits. This is from the original reporter:

 * Calling (9:03)                   -> Calling (8:52) (length)
 * Switch Off (8:46)                -> Switch Off (8:30) (length)
 * Second Shape part I (9:19)       -> Second Shape, Part I (9:02) (length)
 * Risky Nap Under Blue Tree (6:59) -> Risky Nap Under Blue Tree (6:35) (length)
 * Hanging Masses (11:02)           -> Hanging Masses (10:48) (length)
 * Vapor (8:49)                     -> Vapor (8:32) (length)
 * Second Shape part II (9:27)      -> Second Shape, Part II (8:59) (length)
Unmatched tracks (1):
 ! [ Hanging Masses ] | Full Mix (#9) (72:36)

And this is from mine:

 * HoneyBun   -> HoneyBun (id)
 * DeathProof -> DeathProof (id)
Unmatched tracks (1):
 ! Commercial (#9) (3:32)

Hopefully you can spot the similarity. Again, from the original report:

Sending event: database_change
keyfinder: added computed initial key Am for /home/xxx/Downloads/CELL - - Hanging Masses/CELL - - Hanging Masses - - 01 Calling.flac
Sending event: database_change
keyfinder: added computed initial key Am for /home/xxx/Downloads/CELL - - Hanging Masses/CELL - - Hanging Masses - - 02 Switch Off.flac
Sending event: database_change
keyfinder: added computed initial key Db for /home/xxx/Downloads/CELL - - Hanging Masses/CELL - - Hanging Masses - - 03 Second Shape part I.flac
Sending event: database_change
keyfinder: added computed initial key Ebm for /home/xxx/Downloads/CELL - - Hanging Masses/CELL - - Hanging Masses - - 04 Risky Nap Under Blue Tree.flac
Sending event: database_change
keyfinder: added computed initial key Bbm for /home/xxx/Downloads/CELL - - Hanging Masses/CELL - - Hanging Masses - - 05 Hanging Masses.flac
Sending event: database_change
keyfinder: added computed initial key Bbm for /home/xxx/Downloads/CELL - - Hanging Masses/CELL - - Hanging Masses - - 07 Second Shape part II.flac
Sending event: database_change
keyfinder: added computed initial key Ab for /home/xxx/Downloads/CELL - - Hanging Masses/CELL - - Hanging Masses - - 08 Universal Sunrise.flac
Sending event: database_change
keyfinder: added computed initial key Bbm for /home/xxx/Downloads/CELL - - Hanging Masses/CELL - - Hanging Masses - - 09 - Hanging Masses - - Full Mix.flac
Traceback (most recent call last):
  File "/nix/store/7x2zx4d8mf8qfsxgs8ygbl9ba6df5gyi-beets-1.4.3/bin/.beet-wrapped", line 12, in <module>
    sys.exit(main())

And now from mine:

Sending event: import_task_files
mpdqueue: Mere/Solanin/01 Boris.mp3 will be added to queue
mpdqueue: Mere/Solanin/02 HoneyBun.mp3 will be added to queue
mpdqueue: Mere/Solanin/03 Huff.mp3 will be added to queue
mpdqueue: Mere/Solanin/04 Interlewd.mp3 will be added to queue
mpdqueue: Mere/Solanin/05 DeathProof.mp3 will be added to queue
mpdqueue: Mere/Solanin/06 BrokenFlowers.mp3 will be added to queue
mpdqueue: Mere/Solanin/07 Fluff.mp3 will be added to queue
mpdqueue: Mere/Solanin/08 Solanin.mp3 will be added to queue
Traceback (most recent call last):
  File "/usr/local/bin/beet", line 11, in <module>
    sys.exit(main())

Now, I don't know if this is shoddy coding on my part or not, but it seems that whenever these plugins attempt something with these phantom tracks, everything goes horribly wrong and beets goes belly-up.

No issue importing when I disable my own plugin. Tracks 01~08 get copied to the media directory and no crashing.

@sampsyo
Copy link
Member

sampsyo commented Apr 25, 2018

Wow, thank you! This really helps shed some light on the problem. It looks like we need to remove unmatched tracks from the set before we send albums to plugins. I’ll investigate further.

@sampsyo
Copy link
Member

sampsyo commented Apr 29, 2018

Alright! I did some more digging and found the root cause: keyfinder, and your custom plugin, should use task.imported_items() instead of task.items. This gets the items that are actually added to the database, instead of all the ones that were originally found for the import task.

I'm pushing a fix for keyfinder now, but can you please also try that out in your custom plugin?

Hamuko added a commit to Hamuko/beets-mpdqueue that referenced this issue Apr 29, 2018
@Hamuko
Use task.imported_items() instead of task.items to avoid crashes
4e17a08 
beetbox/beets#2537
@Hamuko
Copy link

Hamuko commented Apr 29, 2018

Seems to work fine with task.imported_items(). I can't really remember where I got the task.items, but seeing as the task API is not documented in the plugin documentation, I probably just got it directly from the source code.

@sampsyo
Copy link
Member

sampsyo commented Apr 29, 2018

Good point. If you ever have a moment, we could use a hand explaining this in the plugin authoring docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
needinfo We need more details or follow-up from the filer before this can be tagged "bug" or "feature."
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@sampsyo @Hamuko @jansol