Track when specific plugins last successfully updated an item's metadata

[telejester]

In the course of a large import, remote services used by beets plugins may become unavailable and then reappear, causing errors and uneven results which the user is unable to find and repair.

Beets needs a way to know that a plugin has successfully operated on an asset so that the user may select the affected assets and resubmit then to the plugin for processing.

Any solution should also consider the case where a plugin may be upgraded, which may create a need to select and resubmit assets which were already processed by an older version of the plugin.

In a similar vein, more than one plugin may attempt to modify a standard (i.e., not one that can be namespaced to a given plugin. Lyrics, for instance) metadata field. Which one wins? Can attempted clobbering be detected and presented to the user?

[sampsyo]

This is an interesting direction. I don't have any obvious answers to the questions about how this should work, but we can start working on a design here.

[mister-walter]

As far as clobbering is concerned, maybe plugins should declare a list of fields that they might change before the import process begins? That way, the user could choose to use Plugin A, Plugin B, or "ask each time" for a given field if there was a conflict, and the plugin would change only the fields that it is "allowed" to change.

For the third question, about storing whether or not a plugin has operated on an asset, I can think of two ways to go here:
A. The plugin should handle this, using flexattrs or some other database to store info about processed items
B. Beets automatically adds some attribute to its database every time a plugin is called, and whether or not the plugin successfully ran on the item

Perhaps plugins that run during import should be required to return a value stating whether or not they were able to successfully process an asset, and then Beets would compile a list of failed plugin/asset tuples.

[sampsyo]

I like this general direction a lot. Many—although certainly not all—of beets' plugins have a common structure. They process an album or an item at a time and add extra metadata to them. Every plugin can run either automatically during import or post-facto with an explicit command. I think it's worth supporting this pattern explicitly, and gathering metadata about the process as @mister-walter suggests, so we can add more capabilities to this kind of plugin transparently.

[telejester]

My best guess:

The plugin base class ought to require plugins to state a name and a version and the list of the tag keys it operates on at __init__ time.

It seems reasonable to me for beets to raise a ConfigError exception when it spots a collision, which makes the collision detection a matter of maintaining a dictionary of tags in use.

Regarding how a plugin can mark an item to indicate successful processing, i think the main question there is schema. I'd add a Plugins table which keeps track of names and versions, and a link table to represent the "has processed" predicate with foreign keys back to Item and Plugin.

[arcresu]

The second part of this suggestion is covered by #311. The common "metadata fetcher" plugin helpers can be solved as part of beets' planned plugin refactor (#607). I've renamed this issue to make the focus a little clearer.