Unable to work around issues reported in closed bug report #3674

[dannn-o]

Problem

I do not seem to be able to work around the Python issue reported in #3674 (Name node can't be used with 'None' constant). I've tried all suggestion in that thread and a few others including:

• Installing beets from the Linux Mint repos
• Installing using pip install beets
• Uninstalling that and then using pip install --user beets
• Installing an older version of beets (pip install beets==1.4.9)
• Grabbing the source using Git (git clone https://github.com/beetbox/beets.git), then running python setup.py install (which installed the latest version 1.5.0
• Installing an older version of Python (3.7), and repeating the above step installing from Git.
• Using Python 3.7 with version 1.4.9 downloaded instead of 1.5.0.
• Enabling SNAPD in Linux Mint and installing the beets snap (which installed version 1.4.9)
• I also went back to a virtual machine install of Linux Mint 19.3, which last I knew had been working successfully (but I had not used in many months), and found some so-called "backport" seems to have affected it as well.

Setup

• OS: Linux Mint 20.1 (Ubuntu 20.04)
• Python version: 2.7.18 (python -V), 3.7.10 (python3.7 -V), 3.8.5 (Python3 -V)
• beets version: several (mostly 1.4.9)
• Turning off plugins made problem go away (yes/no): No

My configuration (output of beet config) is:

directory: ~/Music
library: ~/data/musiclibrary.blb

id3v23: yes
plugins: lyrics fetchart zero embedart scrub syncpl

import:
    move: yes
    write: yes
    log: /home/dan/Music/import-log.txt
    timid: yes

zero:
    fields: comments day
    
lyrics:
    force: yes
    
embedart:
    maxwidth: 300
    remove_art_file: yes

syncpl:
    dest: /media/dan/HulkTunes
    playlist_dir: /home/dan/Desktop
    include_playlist: yes
    delete: no
    playlists: HulkTunes

I'm looking for guidance to try and get a working version of beets again. I suspect that installing from Git using the setup.py script has made changes that cannot be undone or uninstalled. Though the specific error text may vary a bit, this is representative of what happens:

beet version
** error loading plugin syncpl:
Traceback (most recent call last):
  File "/home/dan/.local/lib/python3.8/site-packages/beets/plugins.py", line 273, in load_plugins
    namespace = __import__(modname, None, None)
ModuleNotFoundError: No module named 'beetsplug.syncpl'

Traceback (most recent call last):
  File "/home/dan/.local/bin/beet", line 8, in <module>
    sys.exit(main())
  File "/home/dan/.local/lib/python3.8/site-packages/beets/ui/__init__.py", line 1266, in main
    _raw_main(args)
  File "/home/dan/.local/lib/python3.8/site-packages/beets/ui/__init__.py", line 1249, in _raw_main
    subcommands, plugins, lib = _setup(options, lib)
  File "/home/dan/.local/lib/python3.8/site-packages/beets/ui/__init__.py", line 1144, in _setup
    lib = _open_library(config)
  File "/home/dan/.local/lib/python3.8/site-packages/beets/ui/__init__.py", line 1201, in _open_library
    get_path_formats(),
  File "/home/dan/.local/lib/python3.8/site-packages/beets/ui/__init__.py", line 619, in get_path_formats
    path_formats.append((query, template(view.as_str())))
  File "/home/dan/.local/lib/python3.8/site-packages/beets/util/functemplate.py", line 571, in template
    return Template(fmt)
  File "/home/dan/.local/lib/python3.8/site-packages/beets/util/functemplate.py", line 581, in __init__
    self.compiled = self.translate()
  File "/home/dan/.local/lib/python3.8/site-packages/beets/util/functemplate.py", line 614, in translate
    func = compile_func(
  File "/home/dan/.local/lib/python3.8/site-packages/beets/util/functemplate.py", line 155, in compile_func
    prog = compile(mod, '<generated>', 'exec')
ValueError: Name node can't be used with 'None' constant

At this point, my feeling is that it may be easiest to install an older version of Mint (18 or 17) in a virtual machine and try installing beets there. But:

1. I wanted to see if this approach seems sound, and
2. see if there is any easier way to salvage/update my current Mint 20.1 system and get beets going again, even if it's a slightly older version.

Thanks in advance.

[wisp3rwind]

If beets is really running using a version of Python or earlier than 3.8 (i.e 2.7 or 3.7 in your case), I can't see how this issue could still occur. Are you sure that after each of your various steps, you actually executed the newly installed beets from source using the intended version of Python? You should be able to tell from the tracebacks, note the filepaths that are shown, which in the case above includes ~/.local/lib/python3.8/..., i.e. Python 3.8.

Managing various versions of Python in parallel can be quite a mess, unfortunately. Note for example, that this particular 3.8 installation is only for your user, not system-wide.

[dannn-o]

@wisp3rwind - Thanks for your reply. That is exactly why I am reaching out, as one time in the past when I tried to correct an issue with beets, the default Python installation got messed up to the point that the system was rendered unbootable (thankfully, I was able to restore a recent system image to get me up and running again).

I believe version 3.8 is the default for Linux Mint 20.1. The only version I installed was 3.7, and the only step I believe needed Python3.7 specified (correct me if I'm wrong) is executing the setup script (python3.7 setup.py install). As I mentioned above, I believe the initial (default 3.8) installation established references / symbolic pointers involving 3.8 that I just can't work around.

I love the beets app and have been using it for years, but then again it shouldn't be this hard to perform a basic installation that works. I think I may have to go the route I suggested of standing up a virtual machine with the correct environment, but even with that I believe I can use some guidance to make sure I get things correct.

[wisp3rwind]

@wisp3rwind - Thanks for your reply. That is exactly why I am reaching out, as one time in the past when I tried to correct an issue with beets, the default Python installation got messed up to the point that the system was rendered unbootable (thankfully, I was able to restore a recent system image to get me up and running again).

I seriously doubt that simply installing beets caused this, probably something else was changed at the same time, or the Python installation was already in an inconsistent state before. Also, AFAIK, most distros don't need Python to boot.

I believe version 3.8 is the default for Linux Mint 20.1. The only version I installed was 3.7, and the only step I believe needed Python3.7 specified (correct me if I'm wrong) is executing the setup script (python3.7 setup.py install). As I mentioned above, I believe the initial (default 3.8) installation established references / symbolic pointers involving 3.8 that I just can't work around.

Run which beet to figure out which beets installation is first on your PATH(the environment variable which determines paths to look up executables). You'll need to get a clear picture of which versions of Python you have installed into which locations. Each beets installation will be coupled to one Python (since it's basically a Python package). I guess that you've ended up with multiple beets versions on different Python's, and you haven't actually ever run the git version (which can't possibly generate the traceback from your initial post).

[dannn-o]

I seriously doubt that simply installing beets caused this, probably something else was changed at the same time, or the Python installation was already in an inconsistent state before. Also, AFAIK, most distros don't need Python to boot.

It was definitely involving the installation of beets, and in working on this issue I came across someone else who seemed to have pretty much the same problem. I seem to remember someone replying in the issue I opened to use SUDO with PIP, so I suspect it had something to do with that (deadly) combination.

which beet shows /home/dan/.local/bin/beet, and the path is set (correctly, I believe) to #!/usr/bin/python3. As was suggested in the other issue (#3674), I tried changing the "shebang" to point to the Python3.7 installation as @davidheijkamp had suggested, but that offered no relief.

I'm admittedly no coding pro, but the 'master' issue I believe that is the root cause of my issue, #3674, and others was #3621 ("invalid use of AST by beets since Python v3.4 and later."), and that is showing as merged into the master branch in Github since June of last year. I would think that running from master (which I believe is what should have been pulled with my 5th entry in my original post, and showed "1.5.0" as the version installed) would circumvent the "Name node can't be used" issue.

Anyway, after almost 2 weeks without Internet access, the 2nd telephone rep finally and actually fixed the problem with their line today, and "I'm as giddy as a drunken man" (please pardon the Alastair Sim "Christmas Carol" reference), and I can now get the VM environment completed I hope to use as a workaround. Already hitting a few of the issues from back "at a younger age of doing *nix stuff" as was so aptly said in the ITSFOSS link I referenced, like that even Mint 19.3 installs Python 2(.7) version of Python with python-pip, and that along with beets had to be uninstalled and then python3-pip had to be run, etc. I am almost there. I hope to have my config.yaml and a few test albums through in the next few days, and will share the outcome (and general steps for setup, if it works) in case it helps anyone else.

Still open to suggestions of how to remedy the issue with my Mint 20.1 box (that doesn't involve any daring actions that might compromise the default Python for the OS, or isn't more complicated than going the route of using a VM.

[dannn-o]

OK - I had some extra time this evening, so got farther along with the VM route. Install of beets using pip3 install beets went OK, and I successfully copied over my config.yaml file. beet version ran without any errors and indicated the versions I expected:

beets version 1.4.9
Python version 3.6.9
plugins: embedart, fetchart, lyrics, scrub, zero

Good so far. I queued up an album to process and ran the import command. It found the correct match at Musicbrainz, so I pressed "Enter" (defaults to [A]pply). It seemed to continue processing for a bit, but then spit out some errors:

Traceback (most recent call last):
File "/home/dan/.local/bin/beet", line 11, in
sys.exit(main())
File "/home/dan/.local/lib/python3.6/site-packages/beets/ui/init.py", line 1266, in main
_raw_main(args)
File "/home/dan/.local/lib/python3.6/site-packages/beets/ui/init.py", line 1253, in raw_main
subcommand.func(lib, suboptions, subargs)
File "/home/dan/.local/lib/python3.6/site-packages/beets/ui/commands.py", line 955, in import_func
import_files(lib, paths, query)
File "/home/dan/.local/lib/python3.6/site-packages/beets/ui/commands.py", line 925, in import_files
session.run()
File "/home/dan/.local/lib/python3.6/site-packages/beets/importer.py", line 329, in run
pl.run_parallel(QUEUE_SIZE)
File "/home/dan/.local/lib/python3.6/site-packages/beets/util/pipeline.py", line 445, in run_parallel
six.reraise(exc_info[0], exc_info[1], exc_info[2])
File "/home/dan/.local/lib/python3.6/site-packages/six.py", line 719, in reraise
raise value
File "/home/dan/.local/lib/python3.6/site-packages/beets/util/pipeline.py", line 312, in run
out = self.coro.send(msg)
File "/home/dan/.local/lib/python3.6/site-packages/beets/util/pipeline.py", line 194, in coro
func(*(args + (task,)))
File "/home/dan/.local/lib/python3.6/site-packages/beets/importer.py", line 1511, in plugin_stage
func(session, task)
File "/home/dan/.local/lib/python3.6/site-packages/beets/plugins.py", line 143, in wrapper
return func(*args, **kwargs)
File "/home/dan/.local/lib/python3.6/site-packages/beetsplug/lyrics.py", line 827, in imported
False, self.config['force'])
File "/home/dan/.local/lib/python3.6/site-packages/beetsplug/lyrics.py", line 840, in fetch_item_lyrics
lyrics = [self.get_lyrics(artist, title) for title in titles]
File "/home/dan/.local/lib/python3.6/site-packages/beetsplug/lyrics.py", line 840, in
lyrics = [self.get_lyrics(artist, title) for title in titles]
File "/home/dan/.local/lib/python3.6/site-packages/beetsplug/lyrics.py", line 873, in get_lyrics
lyrics = backend.fetch(artist, title)
File "/home/dan/.local/lib/python3.6/site-packages/beetsplug/lyrics.py", line 403, in fetch
return self.lyrics_from_song_api_path(song_api_path)
File "/home/dan/.local/lib/python3.6/site-packages/beetsplug/lyrics.py", line 375, in lyrics_from_song_api_path
lyrics = html.find("div", class="lyrics").get_text()
AttributeError: 'NoneType' object has no attribute 'get_text'

I did a bit of searching through reported Issues, and I believe this matches symptoms in #3535. If I understand correctly, the bug only happens when no lyrics are found (usually I have good luck with popular artists, and the album was OK Computer by Radiohead). I may not get to trying a different album until tomorrow, but would like to get this patch, which was closed 4/17/20, which is after when 1.4.9 was released on 5/30/19. Just trying to avoid making any bad choices again at this juncture before proceeding...I already tried pip3 install -U beets, but that just stayed at v1.4.9.

Any way to force pip3 to install 1.5.0? If not, is there a way to back off everything done installing with pip3, and then use the git clone https://github.com/beetbox/beets.git method to run the setup script from 'Master', which I imagine would require python3 setup.py install be run instead of python setup.py install that is documented in the beets FAQ?

[wisp3rwind]

It was definitely involving the installation of beets, and in working on this issue I came across someone else who seemed to have pretty much the same problem. I seem to remember someone replying in the issue I opened to use SUDO with PIP, so I suspect it had something to do with that (deadly) combination.

What I meant to say is: I doubt that rendering the system unbootable is a problem specific to beets, but rather a more general issue.

I'm admittedly no coding pro, but the 'master' issue I believe that is the root cause of my issue, #3674, and others was #3621 ("invalid use of AST by beets since Python v3.4 and later."), and that is showing as merged into the master branch in Github since June of last year. I would think that running from master (which I believe is what should have been pulled with my 5th entry in my original post, and showed "1.5.0" as the version installed) would circumvent the "Name node can't be used" issue.

I'm repeating myself: I think you did not end up actually running the version from master: Even if it didn't fix the "invalid use of ..." issue, the line numbers in the traceback from your initial post would be different.

You pulled the correct version in the 5th bullet point, and installed it with python setup.py install, but I can't reconstruct from the details you gave which python was first on the PATH by that time. I'm not sure what happens when installing multiple versions of beets as a user, the interesting question being whether /home/dan/.local/bin/beet would be overwritten by the last install or continue pointing to the first.

Any way to force pip3 to install 1.5.0? If not, is there a way to back off everything done installing with pip3, and then use the git clone https://github.com/beetbox/beets.git method to run the setup script from 'Master', which I imagine would require python3 setup.py install be run instead of python setup.py install that is documented in the beets FAQ?

No, 1.5.0 can only be obtained through git clone and setup.py install since it's not a proper release, but only a development version. I fear there might not be a direct way to remove this 1.5.0 install then, but simply installing any other version of beets on top of it should do the trick.

[dannn-o]

@wisp3rwind I'm not sure how I would have ended up with multiple versions of beets installed since I did my best to uninstall before proceeding to the next bullet point. The method using the setup.py script didn't present any way to do this, of course. If there weren't already desperate needs for higher priority items like skilled maintainers for beets, I'd open a suggestion that setup.py be enhanced to offer an "-uninstall" option (or, an even better approached would be a standalone cleanup utility to help ensure a clean slate).

Getting back to my testing with the VM...unfortunately, the issue I posted above ("Nonetype object during lyrics fetch") appears to happen no matter the album. Found something online that seems to identify this as a bug with different patches available, but I couldn't find enough other info to help pursue further. Just for kicks, I uninstalled 1.4.9 and used Pip to install an older version (1.4.6). But, that didn't seem to be enough to avoid the issue. So, I think I've gone as far down the timesuck black hole as I can afford to for at least a little while. Will watch for any other suggestions....again, looking for a clear, simple method to get any somewhat-current version of beets installed (assuming a freshly installed Linux OS - version again negotiable) and successfully working.

[dannn-o]

Some positive progress to report: I think I've hit on a beets installation in the VM that seems to work successfully. I started with the most recent version (1.4.9), and

1. Installed using pip3 install beets==1.4.x
2. Ran through an album import and watched for errors
3. If the "Name Node" or "NoneType" errors from above in this thread occurred, then uninstalled using pip3 uninstall beets and retried step 1 with the next older release.

I finally got a different kind of error when I tried got to version 1.4.5 that seemed to involve the Fetchart plugin. I fairly quickly found this already reported in #2504, and as a remedy I went the route of installing ImageMagick (from the Linux Mint repos). After doing that, I reran the import and....no errors!! I then spent some time confirming the tagging was as expected using Puddletag, and everything looks good.

I will wait before a few more albums go through successfully before I fully declare success. But, this is what's currently installed:

$> beet version
beets version 1.4.5
Python version 3.6.9
plugins: embedart, fetchart, lyrics, scrub, zero

Hoping this might help someone else. When I get some time, I may go back and revisit my Mint 20.1 setup to see if I can pull off something similar and ditch the VM altogether.

[dannn-o]

I got Mint 20.1 working with beets, and followed a similar path as with setting up the VM. However, I discovered a feature of PIP that was key in getting it to work.

First, I used Timeshift to revert back to before I had tried all of the installations in my original post. Then, I used the Deadsnakes PPA to install Python 3.6.13. To get PIP to use this version of Python instead of the OS' default version (3.8), I used this command: python3.6 -m pip install beets==1.4.5
After that, I made sure my config file was proper and performed a beet import without issue!

Just for fun, I tried upgrading to the latest (PIP) version using python3.6 -m pip install -U beets, which upgraded successfully to version 1.4.9. However, imports were again failing with

lyrics = html.find("div", class_="lyrics").get_text()
AttributeError: 'NoneType' object has no attribute 'get_text'

Which, I believe is the same issue described in #3535 (closed). So, I reverted back to version 1.4.5, and have done several imports successfully. I do notice that the lyrics plugin is now not as successful as I was used to, and I assume that is because I'm missing some updates/patches, including the fix for #3535. I do have a few other utilities to help fill in that gap, but it's more time consuming than just having beets get the lyrics. I guess I'm stuck until a new release (1.5.0?) is officially made available to PIP installations. Until then, I guess I must wait it out, but am going to close this issue since I was able to get a 'workaround' of sorts put together.