Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MNT: Add a cache to the Transformer creation in transform_points #1918

Merged
merged 1 commit into from
Nov 14, 2021

Conversation

greglucas
Copy link
Contributor

This adds a cache mechanism to creating Transformer objects, which
can be slow when called frequently. Additionally, this requires
adding attributes to the Mercator creation as the x/y limits are
created in the initializer which uses the cache/hash function.

closes #1915

Rationale

Implications

@Bob131
Copy link

Bob131 commented Oct 31, 2021

Yep, this seems to solve the issue I was hitting.

Here's the benchmark I was using, btw. I wrote it thinking the problem was due to transforming polygon points, but tweaking things it seems the important bits are the grid line labels.

import numpy as np
import matplotlib.pyplot as plt
from matplotlib.animation import FuncAnimation
from matplotlib.patches import Polygon

import cartopy.crs as ccrs


class PolygonAnimation:
    params = [
        ( 8, 128),
        (50, 200)
    ]
    param_names = ['n_points', 'n_frames']

    def setup(self, n_points, n_frames):
        self.polygon_points = \
            np.array([[np.cos(x), np.sin(x)]
                      for x in np.linspace(0, 2 * np.pi, n_points)]) \
                          * 180 / np.pi
        self.proj = ccrs.PlateCarree()
        fig, ax = plt.subplots(subplot_kw=dict(projection=self.proj))
        ax.set_global()
        ax.gridlines(crs=self.proj, draw_labels={'left': 'y'}, x_inline=True)
        self.figure = fig
        self.polygon = ax.add_patch(Polygon(self.polygon_points,
                                            transform=self.proj.as_geodetic()))
        self.animation = FuncAnimation(fig, self.update,
                                       frames=np.linspace(0, 360, n_frames),
                                       blit=True)

    def update(self, frame):
        self.polygon.set_xy(self.polygon_points + (frame, 0))
        return self.polygon,

    def time_draw(self, n_points, n_frames):
        self.animation.save('/tmp/cartopy-bench.gif')
➜  cartopy git:(5540d799) ✗  asv --config benchmarks/asv.conf.json run -b polygon_animation 'HEAD~2..HEAD'
· Creating environments
· Discovering benchmarks
·· Uninstalling from virtualenv-py3.10-fiona-matplotlib-numpy-pykdtree-pyproj-scipy
·· Installing 5540d799 into virtualenv-py3.10-fiona-matplotlib-numpy-pykdtree-pyproj-scipy.
· Running 2 total benchmarks (2 commits * 1 environments * 1 benchmarks)
[  0.00%] · For cartopy commit 5540d799:
[  0.00%] ·· Benchmarking virtualenv-py3.10-fiona-matplotlib-numpy-pykdtree-pyproj-scipy
[ 25.00%] ··· Running (polygon_animation.PolygonAnimation.time_draw--).
[ 50.00%] ··· polygon_animation.PolygonAnimation.time_draw                                                                          ok
[ 50.00%] ··· ========== ============ ============
              --                  n_frames        
              ---------- -------------------------
               n_points       50          200     
              ========== ============ ============
                  8       2.46±0.01s   9.10±0.01s 
                 128      2.74±0.09s   10.2±0.04s 
              ========== ============ ============

[ 50.00%] · For cartopy commit 313f02f3 <master>:
[ 50.00%] ·· Building for virtualenv-py3.10-fiona-matplotlib-numpy-pykdtree-pyproj-scipy.
[ 50.00%] ·· Benchmarking virtualenv-py3.10-fiona-matplotlib-numpy-pykdtree-pyproj-scipy
[ 75.00%] ··· Running (polygon_animation.PolygonAnimation.time_draw--).
[100.00%] ··· polygon_animation.PolygonAnimation.time_draw                                                                          ok
[100.00%] ··· ========== ============ ============
              --                  n_frames        
              ---------- -------------------------
               n_points       50          200     
              ========== ============ ============
                  8       7.13±0.03s   28.6±0.06s 
                 128       7.39±0s     29.5±0.06s 
              ========== ============ ============

dopplershift
dopplershift previously approved these changes Nov 13, 2021
@dopplershift dopplershift dismissed their stale review November 13, 2021 18:02

Decided to hold off.

@dopplershift
Copy link
Contributor

dopplershift commented Nov 13, 2021

@greglucas Could we instead make our own get_transformer_from_crs (or something) which uses lru_cache that we call? I’d really like to avoid monkeypatching pyproj, which can impact people who are using pyproj outside of cartopy within the same program.

@dopplershift
Copy link
Contributor

To be clear, something like:

@lru_cache
def get_transformer_from_crs(crs):
    return Transformer.from_crs(crs)

@greglucas
Copy link
Contributor Author

Yep, good suggestion. Should be updated on both branches now.

This adds a cache mechanism to creating Transformer objects, which
can be slow when called frequently. Additionally, this requires
adding attributes to the Mercator creation as the x/y limits are
created in the initializer which uses the cache/hash function.
@dopplershift dopplershift merged commit 8d1ee5d into SciTools:master Nov 14, 2021
@greglucas greglucas deleted the cache-transformer-crs branch November 14, 2021 14:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

PyProj transformers often reinstantiated
3 participants