Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add hook plugin (fixes #1561) #1603

Merged
merged 26 commits into from
Apr 30, 2016
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
88ece41
Add Hook plugin to run commands on events
jackwilsdon Sep 11, 2015
2d0c217
Improve the way substitute_args is iterated
jackwilsdon Sep 11, 2015
1378351
Add documentation for hook plugin
jackwilsdon Sep 11, 2015
8fea1e6
Add logging for hook plugin
jackwilsdon Sep 11, 2015
0af47bf
Add tests for hook plugin
jackwilsdon Sep 11, 2015
6a56677
Move line break to correct position
jackwilsdon Sep 11, 2015
553dd1f
Add missing colon for configuration
jackwilsdon Sep 11, 2015
276c5da
Remove unused logging import
jackwilsdon Sep 11, 2015
8b7af7f
Fix indentation for wrapped method call
jackwilsdon Sep 11, 2015
46e5f9d
Update tests to follow PEP 8 coding style
jackwilsdon Sep 11, 2015
3d058f4
Changelog for #1603
jackwilsdon Sep 11, 2015
ae2ff61
Use default stdout and stderr streams for hook processes
jackwilsdon Sep 12, 2015
0dff24e
Move Popen call to a single line
jackwilsdon Sep 12, 2015
417a724
Remove unused sys import and use correct platform encoding
jackwilsdon Sep 12, 2015
55bd513
Remove completed TODO comments
jackwilsdon Sep 12, 2015
dd949a9
Merge master back in to fork
jackwilsdon Apr 18, 2016
8b4f349
Improve hook plugin design and configuration
jackwilsdon Apr 18, 2016
686e069
Replace double quotes with single quotes
jackwilsdon Apr 18, 2016
3e35660
Remove unnecessary escaping on double quotes
jackwilsdon Apr 18, 2016
af5ce6e
Fix event name collision in tests and update tests
jackwilsdon Apr 18, 2016
070469e
Remove unicode_literals from __future__ imports
jackwilsdon Apr 18, 2016
028c78a
Update documentation to match new hook plugin design
jackwilsdon Apr 18, 2016
dea091e
Improve error handling for invalid commands
jackwilsdon Apr 18, 2016
ffa2fdd
Merge master back in to fork
jackwilsdon Apr 18, 2016
85fd608
Remove unused dependencies
jackwilsdon Apr 18, 2016
7c9440c
Merge master back in to fork
jackwilsdon Apr 28, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
107 changes: 107 additions & 0 deletions beetsplug/hook.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
# This file is part of beets.
# Copyright 2015, Adrian Sampson.
#
# Permission is hereby granted, free of charge, to any person obtaining
# a copy of this software and associated documentation files (the
# "Software"), to deal in the Software without restriction, including
# without limitation the rights to use, copy, modify, merge, publish,
# distribute, sublicense, and/or sell copies of the Software, and to
# permit persons to whom the Software is furnished to do so, subject to
# the following conditions:
#
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.

"""Allows custom commands to be run when an event is emitted by beets"""
from __future__ import (division, absolute_import, print_function,
unicode_literals)

import subprocess
import sys

from beets.plugins import BeetsPlugin


def create_hook_function(log, event, command, shell, substitute_args):

# TODO: Find a better way of piping STDOUT/STDERR/STDIN between the process
# and the user.
#
# The issue with our current method is that we can only pesudo-pipe
# one (two if we count STDERR being piped to STDOUT) stream at a
# time, meaning we can't have both output and input simultaneously.
# This is due to how Popen.std(out/err) works, as
# Popen.std(out/err).readline() waits until a newline has been output
# to the stream before returning.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it work to just leave off the stdout and stderr arguments to Popen? They'll use the default behavior, which is to share the stdin/stdout/stderr of the parent process.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like removing the stdout and stderr arguments does work, I can't believe I didn't try that earlier. Still can't read from stdin though, which is a pain but I don't think is that big of an issue (interactive hooks would be better as their own plugins anyway). Well it turns out we can! Waiting for the process to exit using process.wait() allows stdin to work too!


# TODO: Find a better way of converting arguments to strings, as I
# currently have a feeling that forcing everything to utf-8 might
# end up causing a mess.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed; we'll need to encode the strings as bytes using the platform's preferred encoding. That's the same encoding we use for decoding arguments, from the _arg_encoding function.


def hook_function(**kwargs):
hook_command = command

for key in substitute_args:
if key in kwargs:
hook_command = hook_command.replace(substitute_args[key],
unicode(kwargs[key],
"utf-8"))

log.debug('Running command {0} for event {1}', hook_command, event)

process = subprocess.Popen(hook_command,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
shell=shell)

while process.poll() is None:
sys.stdout.write(process.stdout.readline())

# Ensure there's nothing left in the stream
sys.stdout.write(process.stdout.readline())

return hook_function


class HookPlugin(BeetsPlugin):
"""Allows custom commands to be run when an event is emitted by beets"""
def __init__(self):
super(HookPlugin, self).__init__()

self.config.add({
'hooks': [],
'substitute_event': '%EVENT%',
'shell': True
})

hooks = self.config['hooks'].get(list)
global_substitute_event = self.config['substitute_event'].get()
global_shell = self.config['shell'].get(bool)

for hook_index in range(len(hooks)):
hook = self.config['hooks'][hook_index]

hook_event = hook['event'].get()
hook_command = hook['command'].get()

if 'substitute_event' in hook:
original = hook['substitute_event'].get()
else:
original = global_substitute_event

if 'shell' in hook:
shell = hook['shell'].get(bool)
else:
shell = global_shell

if 'substitute_args' in hook:
substitute_args = hook['substitute_args'].get(dict)
else:
substitute_args = {}

hook_command = hook_command.replace(original, hook_event)
hook_function = create_hook_function(self._log, hook_event,
hook_command, shell,
substitute_args)

self.register_listener(hook_event, hook_function)
2 changes: 2 additions & 0 deletions docs/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ The new features:
trigger a "lots of music" warning. :bug:`1577`
* :doc:`/plugins/plexupdate`: A new ``library_name`` option allows you to select
which Plex library to update. :bug:`1572` :bug:`1595`
* New :doc:`/plugins/hook` that allows commands to be executed when an event is
emitted by beets. :bug:`1561` :bug:`1603`

Fixes:

Expand Down
77 changes: 77 additions & 0 deletions docs/plugins/hook.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
Hook Plugin
===============

Internally, beets sends events to plugins when an action finishes. These can
range from importing a song (``import``) to beets exiting (``cli_exit``), and
provide a very flexible way to perform actions based on the events. This plugin
allows you to run commands when an event is emitted by beets, such as syncing
your library with another drive when the library is updated.

Hooks are currently run in the order defined in the configuration, however this
is dependent on beets itself (may change in the future) and cannot be controlled
by this plugin.

.. _hook-configuration:

Configuration
-------------

To configure the plugin, make a ``hook:`` section in your configuration
file. The available options are:

- **hooks**: A list of events and the commands to run (see :ref:`individual-hook-configuration`)
Default: Empty.
- **substitute_event**: The string to replace in each command with the name of
the event executing it. This can be used to allow one script to act
differently depending on the event it was called by. Can be individually
overridden (see :ref:`individual-hook-configuration`).
Default: ``%EVENT%``
- **shell**: Run each command in a shell. Can be individually overridden (see
:ref:`individual-hook-configuration`).
Default: ``yes``
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct me if I'm wrong, but we nearly always want shell to be true, right? Unless we want to split shell syntax using shlex ourselves.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After doing some research, I'm not entirely sure. shell is platform dependant, and also depends on the user's $SHELL. After reading this Stack Overflow answer I'm thinking we should make it no by default and let people turn it on if they need it. What do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, not using a shell by default would certainly be more secure! But if we bypass it entirely by default, then a configuration like this:

hook:
  hooks:
    - event: cli_exit
      command: echo "have a nice day!"

won't work (as things currently stand). The plugin will try to find an executable named echo "have a nice day!", which won't do!

We could, however, use shlex, as I hinted before, to parse the arguments. Then we'd get shell-like behavior for arguments formatted intuitively—without actually using a shell. If we do that, then IMO we should remove the shell option altogether; the shlex route would be the only option.

Copy link
Member Author

@jackwilsdon jackwilsdon Apr 18, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So after implementing Popen with shlex, it seems to work as we want. The issue now is the error message that is outputted by Popen when the command doesn't exist isn't exactly "user friendly":

Traceback (most recent call last):
  File "./beet", line 23, in <module>
    beets.ui.main()
  File "/Users/jack/Documents/Development/Personal/beets/beets/ui/__init__.py", line 1238, in main
    _raw_main(args)
  File "/Users/jack/Documents/Development/Personal/beets/beets/ui/__init__.py", line 1228, in _raw_main
    subcommand.func(lib, suboptions, subargs)
  File "/Users/jack/Documents/Development/Personal/beets/beets/ui/commands.py", line 965, in import_func
    import_files(lib, paths, query)
  File "/Users/jack/Documents/Development/Personal/beets/beets/ui/commands.py", line 942, in import_files
    session.run()
  File "/Users/jack/Documents/Development/Personal/beets/beets/importer.py", line 320, in run
    pl.run_parallel(QUEUE_SIZE)
  File "/Users/jack/Documents/Development/Personal/beets/beets/util/pipeline.py", line 347, in run
    self.coro.send(msg)
  File "/Users/jack/Documents/Development/Personal/beets/beets/util/pipeline.py", line 160, in coro
    task = func(*(args + (task,)))
  File "/Users/jack/Documents/Development/Personal/beets/beets/importer.py", line 1400, in manipulate_files
    task.finalize(session)
  File "/Users/jack/Documents/Development/Personal/beets/beets/importer.py", line 543, in finalize
    self._emit_imported(session.lib)
  File "/Users/jack/Documents/Development/Personal/beets/beets/importer.py", line 569, in _emit_imported
    plugins.send('album_imported', lib=lib, album=self.album)
  File "/Users/jack/Documents/Development/Personal/beets/beets/plugins.py", line 458, in send
    result = handler(**arguments)
  File "/Users/jack/Documents/Development/Personal/beets/beets/plugins.py", line 123, in wrapper
    return func(*args, **kwargs)
  File "/Users/jack/Documents/Development/Personal/beets/beetsplug/hook.py", line 54, in hook_function
    subprocess.Popen(command_pieces).wait()
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1335, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

What do you think of catching the OSError and re-throwing it with a better message (i.e. OSError: [Errno 2] No such file or directory: my_command). We could also preserve the stack trace, although it doesn't exactly contain any useful information.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've gone ahead and added the exception modification code in 8b4f349. It's not exactly pretty but it works. The error now looks like this:

Traceback (most recent call last):
  File "./beet", line 23, in <module>
    beets.ui.main()
  File "/Users/jack/Documents/Development/Personal/beets/beets/ui/__init__.py", line 1238, in main
    _raw_main(args)
  File "/Users/jack/Documents/Development/Personal/beets/beets/ui/__init__.py", line 1224, in _raw_main
    subcommands, plugins, lib = _setup(options, lib)
  File "/Users/jack/Documents/Development/Personal/beets/beets/ui/__init__.py", line 1100, in _setup
    plugins = _load_plugins(config)
  File "/Users/jack/Documents/Development/Personal/beets/beets/ui/__init__.py", line 1086, in _load_plugins
    plugins.send("pluginload")
  File "/Users/jack/Documents/Development/Personal/beets/beets/plugins.py", line 458, in send
    result = handler(**arguments)
  File "/Users/jack/Documents/Development/Personal/beets/beets/plugins.py", line 123, in wrapper
    return func(*args, **kwargs)
  File "/Users/jack/Documents/Development/Personal/beets/beetsplug/hook.py", line 61, in hook_function
    subprocess.Popen(command_pieces).wait()
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1335, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory: not_a_real_command

I admit it could still do with some work though, as it's not inherently obvious that it's from the hook plugin.


.. _individual-hook-configuration:

Individual Hook Configuration
-----------------------------

Each element of the ``hooks`` configuration option can be configured separately.
The available options are:

- **event** (required): The name of the event that should cause this hook to execute. See
:ref:`Plugin Events <plugin_events>` for a list of possible values.
- **command** (required): The command to run when this hook executes.
- **substitute_event**: Hook-level override for ``substitute_event`` option in
:ref:`hook-configuration`.
Default: Value of top level ``substitute_event`` option (see :ref:`hook-configuration`)
- **shell**: Hook-level override for ``shell`` option in :ref:`hook-configuration`.
Default: Value of top level ``shell`` option (see :ref:`hook-configuration`).
- **substitute_args**: A key/value set where the key is the name of the an
argument passed to the event (see :ref:`Plugin Events <plugin_events>` for
a list of arguments for each event) and the value is the string to replace
in the command with the value of the argument. Note that any arguments that
are not strings will be converted to strings (e.g. Python objects).
Default: Empty.

Example Configuration
---------------------

.. code-block:: yaml

hook:
hooks:
# Output on exit:
# beets just exited!
# have a nice day!
- event: cli_exit
command: echo "beets just exited!"
- event: cli_exit
command: echo "have a nice day!"

# Output on write
# writing to "<file_name_here>"
# Where <file_name_here> is the file being written to
- event: write
command: echo "writing to \"%FILE_NAME%\""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A somewhat simpler strategy would be to just use ordinary template substitution for the commands. That is, this would just use $path instead of %FILE_NAME% along with a separate mapping from names to tokens. Would that work?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hadn't heard of template substitution, however it does seem to be a much better way of implementing it. I'll try it out now.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So after doing some more work, it looks like template substitution solves half of the problem. What I'd like to do is allow the user to access object attributes, e.g. album.name. I thought template substitution would allow this, however it seems it does not. We may have to write our own parser for this?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm… perhaps we could figure out how to use Python's .format instead. That's what we use for logging: the format is {variable}, but it also allows stuff like {album.title}. Would that work?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added this in 8b4f349, you can now use {event} to access the name of the event and {argument} to access a specific argument (e.g. {album.path}).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! This looks great!

substitute_args:
path: %FILE_NAME%
2 changes: 2 additions & 0 deletions docs/plugins/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ Each plugin has its own set of options that can be defined in a section bearing
ftintitle
fuzzy
freedesktop
hook
ihate
importadded
importfeeds
Expand Down Expand Up @@ -154,6 +155,7 @@ Miscellaneous
a different directory.
* :doc:`duplicates`: List duplicate tracks or albums.
* :doc:`fuzzy`: Search albums and tracks with fuzzy string matching.
* :doc:`hook`: Run a command when an event is emitted by beets.
* :doc:`ihate`: Automatically skip albums and tracks during the import process.
* :doc:`info`: Print music files' tags to the console.
* :doc:`mbcollection`: Maintain your MusicBrainz collection list.
Expand Down
134 changes: 134 additions & 0 deletions test/test_hook.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
# This file is part of beets.
# Copyright 2015, Thomas Scholtes.
#
# Permission is hereby granted, free of charge, to any person obtaining
# a copy of this software and associated documentation files (the
# "Software"), to deal in the Software without restriction, including
# without limitation the rights to use, copy, modify, merge, publish,
# distribute, sublicense, and/or sell copies of the Software, and to
# permit persons to whom the Software is furnished to do so, subject to
# the following conditions:
#
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.

from __future__ import (division, absolute_import, print_function,
unicode_literals)

import os.path
import tempfile

from test import _common
from test._common import unittest
from test.helper import TestHelper

from beets import config
from beets import plugins


def get_temporary_path():
temporary_directory = tempfile._get_default_tempdir()
temporary_name = next(tempfile._get_candidate_names())

return os.path.join(temporary_directory, temporary_name)


# TODO: Find a good way to test shell option
class HookTest(_common.TestCase, TestHelper):
TEST_HOOK_COUNT = 5

def setUp(self):
self.setup_beets() # Converter is threaded

def tearDown(self):
self.unload_plugins()
self.teardown_beets()

def _add_hook(self, event, command, substitute_event=None, shell=None,
substitute_args=None):

hook = {
'event': event,
'command': command
}

if substitute_event is not None:
hook['substitute_event'] = substitute_event

if shell is not None:
hook['shell'] = shell

if substitute_args is not None:
hook['substitute_args'] = substitute_args

hooks = config['hook']['hooks'].get(list) if 'hook' in config else []
hooks.append(hook)

config['hook']['hooks'] = hooks

def test_hook_no_arguments(self):
temporary_paths = [
get_temporary_path() for i in range(self.TEST_HOOK_COUNT)
]

for index, path in enumerate(temporary_paths):
self._add_hook('test_event_{0}'.format(index),
'echo > "{0}"'.format(path))

self.load_plugins('hook')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This actually seems OK to me -- it might look a little weird, of course, that the configuration comes first, but it's logical that the config would need to be in place as early as possible.


for index in range(len(temporary_paths)):
plugins.send('test_event_{0}'.format(index))

for path in temporary_paths:
self.assertTrue(os.path.isfile(path))
os.remove(path)

def test_hook_event_substitution(self):

temporary_directory = tempfile._get_default_tempdir()
event_names = ['test_event_{0}'.format(i) for i in
range(self.TEST_HOOK_COUNT)]

for event in event_names:
self._add_hook(event,
'echo > "{0}"'.format(
os.path.join(temporary_directory, '%EVENT%')
))

self.load_plugins('hook')

for event in event_names:
plugins.send(event)

for event in event_names:
path = os.path.join(temporary_directory, event)

self.assertTrue(os.path.isfile(path))
os.remove(path)

def test_hook_argument_substitution(self):
temporary_paths = [
get_temporary_path() for i in range(self.TEST_HOOK_COUNT)
]

for index, path in enumerate(temporary_paths):
self._add_hook('test_event_{0}'.format(index),
'echo > "%PATH%"'.format(path),
substitute_args={'path': '%PATH%'})

self.load_plugins('hook')

for index, path in enumerate(temporary_paths):
plugins.send('test_event_{0}'.format(index), path=path)

for path in temporary_paths:
self.assertTrue(os.path.isfile(path))
os.remove(path)


def suite():
return unittest.TestLoader().loadTestsFromName(__name__)

if __name__ == b'__main__':
unittest.main(defaultTest='suite')