Lazily fetch album in FormattedItemMapping, it's not needed in most cases

[SimonTeixidor]

Broken out from #3258, as it seems orthogonal.

The item formatter is fetching the related album for each item, even when it is not needed. This PR lazily fetches the album if it is really needed, and does not hit the DB in other cases.

[sampsyo]

So far so good! Maybe we could make the code a little more legible by introducing a simple @lazy_property decorator; something like:

def lazy_property(func):
    field_name = '_' + func.__name__

    @property
    @functools.wraps(func)
    def wrapper(self):
        if hasattr(self, field_name):
            return getattr(self, field_name)

        value = func(self)
        setattr(self, field_name, value)
        return value

    return wrapper

This would work for all three of the properties that are currently manually made lazy, if I'm not mistaken.

[SimonTeixidor]

Neat! I pushed a new commit, introducing lazy_property.

[SimonTeixidor]

I looked into adding some tests, but it seems like test_library.py covers this functionality already.

[sampsyo]

Nice!! This looks great. And I agree that the test coverage is probably already good here. Would you mind doing just a few menial tasks?

• Let’s move the new decorator to util. It could be useful elsewhere.
• A short docstring for the decorator would be nice too.
• Maybe a quick changelog entry would be in order.

Then we can measure the performance impact of this in isolation (both with and without album-level fields) and merge it!

[SimonTeixidor]

Sure, I put lazy_property in util, added the docstring, and added a changelog entry. Here are some quick numbers from my machine:

(virtualenv) ~/beets $ git checkout lazy-album-fetch-in-formatter
M       tox.ini
Switched to branch 'lazy-album-fetch-in-formatter'
Your branch is up to date with 'fork/lazy-album-fetch-in-formatter'.
(virtualenv) ~/beets $ time beet -l /mnt/storage/beets/beets2019-05-05.blb ls | wc -l
14190

real    0m6.156s
user    0m5.929s
sys     0m0.348s
(virtualenv) ~/beets $ time beet -l /mnt/storage/beets/beets2019-05-05.blb ls -f '$artpath' | wc -l
14190

real    0m9.502s
user    0m8.605s
sys     0m1.264s
(virtualenv) ~/beets $ git checkout master
M       tox.ini
Switched to branch 'master'
Your branch is up to date with 'origin/master'.
(virtualenv) ~/beets $ time beet -l /mnt/storage/beets/beets2019-05-05.blb ls | wc -l
14190

real    0m10.440s
user    0m9.498s
sys     0m1.463s
(virtualenv) ~/beets $ time beet -l /mnt/storage/beets/beets2019-05-05.blb ls -f '$artpath' | wc -l
14190

real    0m9.447s
user    0m8.511s
sys     0m1.352s
(virtualenv) ~/beets $

Not a very thorough benchmark, but it looks pretty significant. Queries that don't use album level fields run about 1.5x slower on master compared to on lazy-album-fetch-in-formatter. Queries that do use the album level fields run about the same speed.

[sampsyo]

Nice! I ran my own little performance test here, with three runs for each of the 4 treatments:

command	master	lazy-album-fetch-in-formatter
beet ls	7.21s, 7.10s, 7.08s	4.96s, 3.95s, 3.79s
beet ls -f '$artpath'	6.74s, 6.44s, 6.57s	7.03s, 6.45s, 6.46s

Long story short, that's a 1.7x speedup for the default format and no significant change for the format that includes an album-level field. Wahoo!!!

[sampsyo]

Merged! Just for fun, I did a quick comparison between the new master and the previous release, before all this focus on performance. v1.4.7 listed my database in an average of 13.0 seconds, and master got an average time of 3.9 seconds (across 3 runs each). That comes to a shockingly good 3.4x speedup! 🎉 🐎