Playlist plugin #3145

Holzhaus · 2019-02-15T22:08:19Z

Adds M3U playlist support as a query to beets and thus partially
resolves issue #123. The implementation is heavily based on #2380 by
Robin McCorkell.

It supports referencing playlists by absolute path:

$ beet ls playlist:/path/to/someplaylist.m3u

It also supports referencing playlists by name. The playlist is then
seached in the playlist_dir and the .m3u extension is appended to the
name:

$ beet ls playlist:anotherplaylist

The configuration for the plugin looks like this:

playlist:
    relative_to: library
    playlist_dir: /path/to/playlists

The relative_to option specifies how relative paths in playlists are
handled. By default, paths are relative to the library directory. It
also possible to make them relative to the playlist or set the option
or set it to a fixed path.

Holzhaus · 2019-02-15T22:18:11Z

One of the main problems is that the query is rather slow with large libraries. Since key in model_cls._fields evaluates to False, the col_clause() implemention will not be used. Any ideas?

sampsyo

Cool! This is a really neat prototype.

I think the issue you're running into with the query is a bit more fundamental than it might seem at first: field-qualified queries like foo:bar are currently quite intrinsically tied to the existence of a field called foo (either built-in or flexible). You're introducing a type for a "fake" field called playlist here, which works but isn't really what the query system is expecting. Because PlaylistQuery is a FieldQuery, the system really wants to do a playlist-style query of some field (and that field, here, is always called playlist).

I agree that the syntax playlist:foo would be desirable for this plugin. So maybe it's worth digging a little bit into building special support for these "pseudo-fields" for invoking special queries with field-like syntax but without querying actual, underlying fields. For what it's worth, the random plugin (which currently does not define a query) could use this too: a query term like random:5 could use the same mechanism.

sampsyo · 2019-02-16T15:47:17Z

beets/autotag/match.py

@@ -345,7 +345,7 @@ def _sort_candidates(candidates):
    return sorted(candidates, key=lambda match: match.distance)


-def _add_candidate(items, results, info):
+def _add_candidate(items, results, info, force=False):


This addition of the force parameter seems to be an unrelated change that got included in this PR by mistake?

Yeah, sorry about that.

sampsyo · 2019-02-16T15:50:30Z

beetsplug/playlist.py

+        config = beets.config['playlist']
+
+        # Get the full path to the playlist
+        if os.path.isabs(beets.util.syspath(pattern)):


I might recommend using isfile. That way, anything that refers to an actual file on disk will work, and if the file doesn't exist, we'll fall back on searching playlist_dir.

That would introduce a race condition in case the file is deleted between the isfile and the open call. With proper error handling this wouldn't be too bad, but I restructured the code a bit so that isfile is not needed at all.

Looks perfect. 👍

sampsyo · 2019-02-16T15:54:50Z

beetsplug/playlist.py

+        relative_to = beets.util.bytestring_path(relative_to)
+
+        self.paths = []
+        with open(beets.util.syspath(playlist_path), 'rb') as f:


It might be nice to have some error handling here in case the playlist does not exist (either as a plain filename or in the centralized playlist directory).

Done. It would be nice to print some error message to the console, but I don't know how. I don't really want to assign the self._log attribute of PlaylistPlugin to the PlaylistQuery class, but I don't see another way to do this.

Good point. Threading through the plugin-specific log object here seems too hard. Maybe we can work something out as part of the more general "pseudo-field" query infrastructure.

sampsyo · 2019-02-16T15:57:30Z

beetsplug/playlist.py

+            # Playlist is empty
+            return '0', ()
+        clause  = 'BYTELOWER(path) IN ({0})'.format(
+            ', '.join('BYTELOWER(?)' for path in self.paths))


I note that this query is always case-insensitive. That's probably fine, but it's probably worth noting, at least in the docstring for this class. (See the built-in PathQuery if you're curious—it can either be case-sensitive or case-insensitive, depending on the filesystem.)

Well, the problem is that Playlists can contain multiple entries from different filesystems. I should probably make this case sensitive - case-insensitivity is more a convenience feature for FAT/NTFS users and not actually needed.

Also a good point! Case-sensitive sounds fine; we can revisit this if it gets inconvenient.

sampsyo · 2019-02-16T15:59:22Z

docs/plugins/playlist.rst

+- **relative_to**: Interpret paths in the playlist files relative to a base
+  directory. It is also possible to set it to ``playlist`` to use the
+  playlist's parent directory as base directory.
+  Default: ``library``


It's probably a good idea to mention explicitly that there are three options here: playlist, library, or a path.

Adds M3U playlist support as a query to beets and thus partially resolves issue beetbox#123. The implementation is heavily based on beetbox#2380 by Robin McCorkell. It supports referencing playlists by absolute path: $ beet ls playlist:/path/to/someplaylist.m3u It also supports referencing playlists by name. The playlist is then seached in the playlist_dir and the ".m3u" extension is appended to the name: $ beet ls playlist:anotherplaylist The configuration for the plugin looks like this: playlist: relative_to: library playlist_dir: /path/to/playlists The relative_to option specifies how relative paths in playlists are handled. By default, paths are relative to the "library" directory. It also possible to make them relative to the "playlist" or set the option or set it to a fixed path.

Implement the col_clause method for faster, sqlite-based querying. This will only make a difference if the "fast" kwarg is set to True.

Holzhaus · 2019-02-17T15:20:05Z

For some reason some testcase on Windows test keep throwing errors:
https://ci.appveyor.com/project/beetbox/beets/builds/22435461/job/prrm5wikfjp8xvm2#L122

Might be related to spaces in the path, but I'm not sure and I can't fix this since I don't have Windows. The other tests should work fine now.

Holzhaus · 2019-02-17T16:24:10Z

Looks like spaces in the path were indeed the problem. Should work now.

The performance issue could be fixed via #3149. If that PR is merged, we only need to override the check_fast method in PlaylistQuery with this one:

@classmethod
def check_fast(self, key, model_cls):
    return issubclass(model_cls, beets.library.Item)

Holzhaus · 2019-02-17T17:02:25Z

Is anything else missing? Should I squash the commits?

sampsyo · 2019-02-17T18:16:15Z

Awesome! This looks perfect. Thank you for tracking down the Windows problem even without access to a Windows machine—that was heroic. ✨

No need for a squash from my perspective. I'll merge this now. 🚀

Playlist plugin

As discussed here: #3145 (review) This would replace the need for #3149.

Holzhaus force-pushed the playlist-plugin branch from 2ceb8ef to 9887f1e Compare February 15, 2019 23:03

sampsyo reviewed Feb 16, 2019

View reviewed changes

Holzhaus added 4 commits February 17, 2019 14:38

playlist: Improve speed in PlaylistQuery class

19b92e1

Implement the col_clause method for faster, sqlite-based querying. This will only make a difference if the "fast" kwarg is set to True.

docs: Add documentation for the playlist plugin

cc501be

docs: Add playlist plugin to the changelog

d78bade

Holzhaus force-pushed the playlist-plugin branch from 83ef4dc to d52dcdd Compare February 17, 2019 14:11

Holzhaus added 8 commits February 17, 2019 15:17

test: Add test suite for the playlist plugin

0988a2a

playlist: Restructure playlist reading code and add error handling

f9f2fa0

test: Add playlist testcases for nonexisting playlists

d52dcdd

docs: Reword documentation of playlist plugin's relative_to option

34cdeee

test: Get rid of TemporaryDirectory to restore Python 2.7 compatibility

d4039be

test: Don't use unix-only paths in playlist plugin testcase

32b6df0

playlist: Also catch IOErrors to restore Python 2.7 compatiblity

055f2d3

test: Fix playlist plugin path handling for Windows compatibility

31c687c

test: Ensure path quoting in playlist tests

d6022e2

Holzhaus mentioned this pull request Feb 17, 2019

Allow overriding "fast" detection in FieldQuery subclasses #3149

Closed

playlist: Restore case sensitivity in col_clause method

4f1a468

sampsyo merged commit 4f1a468 into beetbox:master Feb 17, 2019

sampsyo added a commit that referenced this pull request Feb 17, 2019

Merge pull request #3145 from Holzhaus/playlist-plugin

00667bd

Playlist plugin

sampsyo added a commit that referenced this pull request Feb 17, 2019

Tiny doc refinements for #3145

3633c1e

sampsyo added a commit that referenced this pull request Feb 17, 2019

Prototype support for named (pseudo-field) queries

d8e1676

As discussed here: #3145 (review) This would replace the need for #3149.

sampsyo mentioned this pull request Feb 17, 2019

Add support for "named queries" and use them in the playlist plugin #3150

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Playlist plugin #3145

Playlist plugin #3145

Holzhaus commented Feb 15, 2019

Holzhaus commented Feb 15, 2019

sampsyo left a comment

sampsyo Feb 16, 2019

Holzhaus Feb 17, 2019

sampsyo Feb 16, 2019

Holzhaus Feb 17, 2019

sampsyo Feb 17, 2019

sampsyo Feb 16, 2019

Holzhaus Feb 17, 2019

sampsyo Feb 17, 2019

sampsyo Feb 16, 2019

Holzhaus Feb 17, 2019

sampsyo Feb 17, 2019

sampsyo Feb 16, 2019

Holzhaus Feb 17, 2019

Holzhaus commented Feb 17, 2019 •

edited

Loading

Holzhaus commented Feb 17, 2019

Holzhaus commented Feb 17, 2019

sampsyo commented Feb 17, 2019

Playlist plugin #3145

Playlist plugin #3145

Conversation

Holzhaus commented Feb 15, 2019

Holzhaus commented Feb 15, 2019

sampsyo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Holzhaus commented Feb 17, 2019 • edited Loading

Holzhaus commented Feb 17, 2019

Holzhaus commented Feb 17, 2019

sampsyo commented Feb 17, 2019

Holzhaus commented Feb 17, 2019 •

edited

Loading