Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up ZODB.blob.BushyLayout.oid_to_path on Python 3 #161

Merged
merged 1 commit into from
Apr 14, 2017

Conversation

jamadden
Copy link
Member

Profiling (zodb/zodbshootout#32) showed that
this method was the only blob-related method that showed up in a test
of creating blobs, other than those that actually performed IO.

With this change its total and cumulative time drops from 0.003/0.004
to 0.001/0.002 in a small benchmark. Blobs created per second shows a
small but consistent improvement.

Before:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      100    0.005    0.000    0.005    0.000 {built-in method rename}
      100    0.004    0.000    0.004    0.000 {function BlobFile.close at 0x1080d3a60}
      200    0.003    0.000    0.004    0.000 blob.py:576(oid_to_path)
      101    0.003    0.000    0.003    0.000 {built-in method mkdir}
      100    0.002    0.000    0.002    0.000 blob.py:333(__init__)
      402    0.002    0.000    0.005    0.000 {method 'dump' of '_pickle.Pickler' objects}
        1    0.002    0.002    0.034    0.034 Connection.py:553(_store_objects)
      201    0.002    0.000    0.002    0.000 {built-in method stat}
     5633    0.001    0.000    0.002    0.000 {built-in method isinstance}

After:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      100    0.005    0.000    0.005    0.000 {built-in method rename}
      101    0.005    0.000    0.005    0.000 {built-in method mkdir}
      100    0.004    0.000    0.004    0.000 {function BlobFile.close at 0x10636aa60}
      402    0.002    0.000    0.005    0.000 {method 'dump' of '_pickle.Pickler' objects}
      100    0.002    0.000    0.002    0.000 blob.py:333(__init__)
        1    0.002    0.002    0.035    0.035 Connection.py:553(_store_objects)
      201    0.002    0.000    0.002    0.000 {built-in method stat}
     4033    0.001    0.000    0.001    0.000 {built-in method isinstance}
   ....
      200    0.001    0.000    0.002    0.000 blob.py:576(oid_to_path)

Profiling (zodb/zodbshootout#32) showed that
this method was the only blob-related method that showed up in a test
of creating blobs, other than those that actually performed IO.

With this change its total and cumulative time drops from 0.003/0.004
to 0.001/0.002 in a small benchmark. Blobs created per second shows a
small but consistent improvement.

Before:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      100    0.005    0.000    0.005    0.000 {built-in method rename}
      100    0.004    0.000    0.004    0.000 {function BlobFile.close at 0x1080d3a60}
      200    0.003    0.000    0.004    0.000 blob.py:576(oid_to_path)
      101    0.003    0.000    0.003    0.000 {built-in method mkdir}
      100    0.002    0.000    0.002    0.000 blob.py:333(__init__)
      402    0.002    0.000    0.005    0.000 {method 'dump' of '_pickle.Pickler' objects}
        1    0.002    0.002    0.034    0.034 Connection.py:553(_store_objects)
      201    0.002    0.000    0.002    0.000 {built-in method stat}
     5633    0.001    0.000    0.002    0.000 {built-in method isinstance}

After:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      100    0.005    0.000    0.005    0.000 {built-in method rename}
      101    0.005    0.000    0.005    0.000 {built-in method mkdir}
      100    0.004    0.000    0.004    0.000 {function BlobFile.close at 0x10636aa60}
      402    0.002    0.000    0.005    0.000 {method 'dump' of '_pickle.Pickler' objects}
      100    0.002    0.000    0.002    0.000 blob.py:333(__init__)
        1    0.002    0.002    0.035    0.035 Connection.py:553(_store_objects)
      201    0.002    0.000    0.002    0.000 {built-in method stat}
     4033    0.001    0.000    0.001    0.000 {built-in method isinstance}
   ....
      200    0.001    0.000    0.002    0.000 blob.py:576(oid_to_path)
@jimfulton jimfulton merged commit 3580ddd into master Apr 14, 2017
@jimfulton
Copy link
Member

Thanks!

@jimfulton jimfulton deleted the faster-bush-oid-to-path branch April 14, 2017 15:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants