Implement the basic AcousticBrainz Submit plugin #2342

inytar · 2016-12-23T22:44:00Z

This is a new upload for #2340 as that had some strange problems with the pull request and trying to fix it closed the pull request.

Copy of the last comment in that pull request:

Thanks for all the advice! I updated the pull request to hopefully fix the strange problem with authorization/ect. I'll also look into the comments, documentation and tests.
Some things I'm wondering:

Do we want to raise an error if analysis or submitting a file fails?

Do we want to save that a file has been analyzed and submitted? The analysis takes a long time and should not change if done multiple times and as far as I can see the the server code ignores any upload that is exactly the same as one already in the system.

ghost · 2016-12-23T23:01:20Z

this failure here https://travis-ci.org/beetbox/beets/jobs/186448711 is the same I'm seeing in my PR. something has changed in either mutagen or the underlying build env.

ghost · 2016-12-24T00:27:24Z

it seems to be mutagen. I filed #2343

sampsyo · 2016-12-24T01:34:27Z

Thanks for opening the new PR! This one seems to have cleared up the weird problems.

Some answers to your questions from the other thread:

Do we want to raise an error if analysis or submitting a file fails?

I think the appropriate thing to do is to use self._log.error to report the error, but no need to stop the whole process. This is usually nice because it means that if things go wrong for one specific file, or if the network connection drops out intermittently, progress isn't lost.

Do we want to save that a file has been analyzed and submitted? The analysis takes a long time and should not change if done multiple times and as far as I can see the the server code ignores any upload that is exactly the same as one already in the system.

Good idea! If you don't mind, let's add this in a subsequent PR—we can merge the first version and then add this "don't submit twice" feature. It should be pretty easy—we can use an abz_submitted flexible attribute or somesuch, or even store the full analysis results—but we can get the basic functionality working first.

A UserError is now raised if the plugin can be started. If an item fails to be analyzed or be submited an error is logged and the next item is tried.

inytar · 2016-12-26T21:55:08Z

I've updated this pull request, hopefully worked all the comments from the last pull request and added documentation.

sampsyo · 2016-12-26T23:12:09Z

Wonderful; thank you! ✨

It looks like Travis is reminding us that the new file needs the standard header, complete with license text and the coding comment at the top.

sampsyo · 2016-12-26T23:20:22Z

Or! There's some argument to be made here that we should just add this functionality to the acousticbrainz plugin, which has the same requirements (i.e., just requests). That would be neat and tidy!

sampsyo

I made a couple of small suggestions. It's definitely time to call this GCI task "completed," though, if you'd like to submit it!

sampsyo · 2016-12-26T23:14:04Z

beetsplug/absubmit.py

+                )
+        else:
+            # Implicit path to extractor, search for it in path
+            # TODO how to check for on Windows?


Is this comment still necessary? Any more questions about Windows?

sampsyo · 2016-12-26T23:21:10Z

docs/plugins/absubmit.rst

+Installation
+------------
+
+The `absubmit` plugin requires the the `streaming_extractor_music`_ program to run. Its source can be found on `github`_, and while it is possible to compile the extractor from source, AcousticBrainz would prefer if you used thier binary (see the AcousticBrainz `FAQ`_).


Capitalize "GitHub"

the the => the

sampsyo · 2016-12-26T23:21:43Z

docs/plugins/absubmit.rst

+AcousticBrainz Submit Plugin
+============================
+
+The `absubmit` plugin uses the `streaming_extractor_music`_ program to analyse an audio file and calculate different acoustic properties of the audio, the plugin then uploads this metadata to the AcousticBrainz server. The plugin does this when calling the ``beet absumbit [QUERY]`` command or on importing if the `auto` configuration options is set to ``yes``.


The first comma here should actually end the sentence (and begin a new one).

…acoustic properties of the audio. The plugin then uploads…

sampsyo · 2016-12-26T23:23:03Z

docs/plugins/absubmit.rst

+
+    pip install requests
+
+After installing both the extractor binary and request you can enable the plgin ``absubmit`` in your configuration (see :ref:`using-plugins`).


plgin -> plugin

and request => and requests

SusannaMaria · 2016-12-27T13:55:43Z

Have added force option and check if acousticbrainz tag already set, similar to the acousticbrainz plugin pull request from myself
SusannaMaria@c474142

SusannaMaria · 2016-12-27T14:12:21Z

There is some problem with removing the temp-json file in windows10.

   File "C:\Python27\lib\site-packages\beets-1.4.3-py2.7.egg\beetsplug\absubmit.py", line 149, in _get_analysis
os.remove(filename)
   WindowsError: [Error 32] Der Prozess kann nicht auf die Datei zugreifen, da sie von einem anderen Prozess verwendet wird: 'c:\\users\\susanna\\appdata\\local\\temp\\tmpem70wo.json'

Sorry the the german error text, it says the file cannot deleted because ifs used from a different process.

Will check with https://docs.python.org/3/library/tempfile.html
it should solve this issue, maybe its completely windows related.

Kraymer · 2016-12-27T15:51:25Z

docs/plugins/absubmit.rst

@@ -0,0 +1,31 @@
+AcousticBrainz Submit Plugin


options is => option is

Kraymer · 2016-12-27T15:52:57Z

docs/plugins/absubmit.rst

+Configuration
+-------------
+
+To configute the plugin, make a ``absubmit:`` section in your configuration file. The available options are:


configute => configure

sampsyo · 2016-12-27T17:55:12Z

Thanks, @SusannaMaria—perhaps we're doing something wrong when closing the new temporary file?

SusannaMaria · 2016-12-27T21:42:59Z

Have to check tomorrow, just finished some changes because of force-option on @inytar fork (made a pull request on his fork) inytar@b8fa8dc

inytar · 2016-12-31T19:16:00Z

Sorry for the long wait, was away for a couple of days. I've fixed the above mentioned comments.
@sampsyo would you like to submit the plugin as is, or prefer it being merged with the acousticbrainz plugin? The reasons I originally did not do this are: this plugin needs a new dependency that is not needed for the original acousticbrainz plugin; this plugin does not save anything to file, but just upoads it to acousticbrainz.
@SusannaMaria where you albe to fix the temp file problem in your branch?

sampsyo · 2016-12-31T23:03:03Z

No problem at all! There's no rush here. 😃

Sure—on further reflection, I can see the argument for keeping these as separate plugins. For the other one, the documentation doesn't even need to mention that you need to install the analysis program.

It looks like Travis is still complaining about the coding line at the top of the file—and indeed, the new Python file needs the entire standard header comment. Other than that, I think this is ready to merge.

sampsyo · 2016-12-31T23:00:29Z

docs/plugins/absubmit.rst

+Installation
+------------
+
+The `absubmit` plugin requires the `streaming_extractor_music`_ program to run. Its source can be found on `GitHub`_, and while it is possible to compile the extractor from source, AcousticBrainz would prefer if you used thier binary (see the AcousticBrainz `FAQ`_).


thier -> their

Implement the basic AcousticBrainz Submit plugin

sampsyo · 2017-01-01T05:48:27Z

This is all merged up! ✨ Thank you for all your effort on this, @inytar.

tux-00 · 2017-01-01T20:28:54Z

Maybe I'm wrong but the documentation says "To submit audio to AcousticBrainz, files must be tagged with MusicBrainz Identifiers. If they are not tagged, we recommend Picard.", maybe it's a good idea to write this sentence in docs/plugins/absubmit.rst ?

sampsyo · 2017-01-02T00:26:09Z

@tux-00 Sure; it's a good point to mention that music needs MBIDs for submission. Would you mind adding that to the docs with a PR? It's just a matter of hitting the "edit" button on the appropriate page on GitHub.

And we clearly shouldn't say "we recommend Picard" because beets of course does this perfectly well. Under ordinary circumstances, users' music will have already had MBIDs added by the import process.

tux-00 · 2017-01-02T12:39:44Z

No problem but just to make sure that I understand :

What plugin generates the MBID of a track ? absubmit ?

I'm a little bit confused with the names of the beets MusicBrainz plugins. For example the plugin "MusicBrainz Submit Plugin" does not seem to submit anything. It's possible that I don't understand something, I'm new with MusicBrainz !

sampsyo · 2017-01-02T15:05:36Z

The importer (with no plugin at all) adds track MBIDs.

And that mbsumbit plugin generates text that you can paste into a submission form. That's the best we can do, unfortunately, because MusicBrainz doesn't offer a programmatic API for submission.

And no worries at all! I'm happy to answer questions, even if they seem basic.

tux-00 · 2017-01-02T17:51:15Z

When trying to run beets from sources, if the plugin doesn't find the the streaming_extractor_music binary beets raises this error :

Traceback (most recent call last):
  File "/usr/bin/beet", line 9, in <module>
    load_entry_point('beets==1.4.3', 'console_scripts', 'beet')()
  File "/home/tux/git/beets/beets/ui/__init__.py", line 1209, in main
    _raw_main(args)
  File "/home/tux/git/beets/beets/ui/__init__.py", line 1192, in _raw_main
    subcommands, plugins, lib = _setup(options, lib)
  File "/home/tux/git/beets/beets/ui/__init__.py", line 1089, in _setup
    plugins = _load_plugins(config)
  File "/home/tux/git/beets/beets/ui/__init__.py", line 1075, in _load_plugins
    plugins.send("pluginload")
  File "/home/tux/git/beets/beets/plugins.py", line 451, in send
    for handler in event_handlers()[event]:
  File "/home/tux/git/beets/beets/plugins.py", line 434, in event_handlers
    for plugin in find_plugins():
  File "/home/tux/git/beets/beets/plugins.py", line 288, in find_plugins
    _instances[cls] = cls()
  File "/home/tux/git/beets/beetsplug/absubmit.py", line 84, in __init__
    self.extractor = distutils.spawn.find_executable(self.extractor)
AttributeError: 'module' object has no attribute 'spawn'

So if I understand, if the importer cannot find the musicbrainz track id I cannot use the acousticbrainz to tag my tracks.
Is that right ?

sampsyo · 2017-01-02T18:32:33Z

Hmm; that actually looks like an unrelated bug—I don't know why distutils.spawn can't be found, but this doesn't have to do with MBIDs.

EDIT: Oh, I see—those were two different questions. Right; the importer has to get an MBID for your music in order for AB submission to work.

sampsyo · 2017-01-02T18:35:33Z

In fact, could you please open a separate bug with that traceback and full details about your setup? Looks like we need to investigate more closely.

tux-00 · 2017-01-02T20:14:09Z

@sampsyo yes I will ! :)

tux-00 · 2017-01-03T17:37:12Z

It's strange, I can't reproduce the distutils error.. Now I get a good error reporting :
error: Extractor command does not exist: /usr/bin/test.

I remember that I have played with pip to install beets from sources, maybe something happened and reboot solve the problem. This is the only reason I've found !

sampsyo · 2017-01-03T19:30:10Z

Weird! Well, let us know if it comes back.

inytar added 2 commits December 23, 2016 17:27

Implement the basic AcousticBrainz Submit plugin

fd3ff91

Move extractor sha calculation to pluging init.

f61aa7a

inytar added 4 commits December 26, 2016 16:05

Update errors and logging of ABSubmit plugin

b861870

A UserError is now raised if the plugin can be started. If an item fails to be analyzed or be submited an error is logged and the next item is tried.

Update comments for clarification

1465167

Add documentation for the absubmit plugin

90ea842

Add absubmit plugins dependencies to the extras_require field

d0dd0b2

sampsyo reviewed Dec 26, 2016

View reviewed changes

Kraymer reviewed Dec 27, 2016

View reviewed changes

docs/plugins/absubmit.rst

@@ -0,0 +1,31 @@

AcousticBrainz Submit Plugin

Copy link

Contributor

Kraymer Dec 27, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

options is => option is

Kraymer reviewed Dec 27, 2016

View reviewed changes

abdullahzen approved these changes Dec 30, 2016

View reviewed changes

inytar added 2 commits December 31, 2016 14:11

Remove fixed TODO comment

cfe9c0f

Fix typos in absubmit documentation

9ab67d5

sampsyo reviewed Dec 31, 2016

View reviewed changes

inytar added 2 commits December 31, 2016 18:26

More spelling corrections for the absubmit plugin documentation

1a4c74a

Add header to the absubmit plugin

b57b3f7

sampsyo merged commit b57b3f7 into beetbox:master Jan 1, 2017

sampsyo added a commit that referenced this pull request Jan 1, 2017

Merge pull request #2342 from inytar/master

4c1b8c2

Implement the basic AcousticBrainz Submit plugin

sampsyo added a commit that referenced this pull request Jan 1, 2017

Changelog entry for #2342 (fix #2253)

32dc808

tux-00 mentioned this pull request Jan 2, 2017

Add notes about AcousticBrainz requirements #2364

Merged

This was referenced Jan 2, 2017

Absubmit add section on skipped file by the plugin inytar/beets#2

Closed

Absubmit add section in the documentation on skipped file by the plugin #2365

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement the basic AcousticBrainz Submit plugin #2342

Implement the basic AcousticBrainz Submit plugin #2342

inytar commented Dec 23, 2016

ghost commented Dec 23, 2016

ghost commented Dec 24, 2016

sampsyo commented Dec 24, 2016

inytar commented Dec 26, 2016

sampsyo commented Dec 26, 2016

sampsyo commented Dec 26, 2016

sampsyo left a comment

sampsyo Dec 26, 2016

sampsyo Dec 26, 2016

Kraymer Dec 27, 2016

sampsyo Dec 26, 2016

sampsyo Dec 26, 2016

Kraymer Dec 27, 2016

SusannaMaria commented Dec 27, 2016

SusannaMaria commented Dec 27, 2016

Kraymer Dec 27, 2016

Kraymer Dec 27, 2016

sampsyo commented Dec 27, 2016

SusannaMaria commented Dec 27, 2016

inytar commented Dec 31, 2016

sampsyo commented Dec 31, 2016

sampsyo Dec 31, 2016

sampsyo commented Jan 1, 2017

tux-00 commented Jan 1, 2017 •

edited

Loading

sampsyo commented Jan 2, 2017

tux-00 commented Jan 2, 2017

sampsyo commented Jan 2, 2017

tux-00 commented Jan 2, 2017

sampsyo commented Jan 2, 2017 •

edited

Loading

sampsyo commented Jan 2, 2017

tux-00 commented Jan 2, 2017

tux-00 commented Jan 3, 2017

sampsyo commented Jan 3, 2017


		pip install requests

		After installing both the extractor binary and request you can enable the plgin ``absubmit`` in your configuration (see :ref:`using-plugins`).

Implement the basic AcousticBrainz Submit plugin #2342

Implement the basic AcousticBrainz Submit plugin #2342

Conversation

inytar commented Dec 23, 2016

ghost commented Dec 23, 2016

ghost commented Dec 24, 2016

sampsyo commented Dec 24, 2016

inytar commented Dec 26, 2016

sampsyo commented Dec 26, 2016

sampsyo commented Dec 26, 2016

sampsyo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SusannaMaria commented Dec 27, 2016

SusannaMaria commented Dec 27, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sampsyo commented Dec 27, 2016

SusannaMaria commented Dec 27, 2016

inytar commented Dec 31, 2016

sampsyo commented Dec 31, 2016

Choose a reason for hiding this comment

sampsyo commented Jan 1, 2017

tux-00 commented Jan 1, 2017 • edited Loading

sampsyo commented Jan 2, 2017

tux-00 commented Jan 2, 2017

sampsyo commented Jan 2, 2017

tux-00 commented Jan 2, 2017

sampsyo commented Jan 2, 2017 • edited Loading

sampsyo commented Jan 2, 2017

tux-00 commented Jan 2, 2017

tux-00 commented Jan 3, 2017

sampsyo commented Jan 3, 2017

tux-00 commented Jan 1, 2017 •

edited

Loading

sampsyo commented Jan 2, 2017 •

edited

Loading