-
Notifications
You must be signed in to change notification settings - Fork 609
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
perf(dot-columns): speed up heavily used .columns
#9921
Conversation
The difference in performance is pretty stark:
Sure, we're in microsecond territory, but this property is used quite a bit, and hit pretty heavily in selector processing, which tends to invoke |
Another approach is to replace the use of The downside there is that with |
Moving to draft to avoid merging before 9.4 |
6d12c19
to
d938306
Compare
6c6ea0d
to
a10640e
Compare
a10640e
to
881ef26
Compare
The 3 cloud tests that were failing are now passing.
|
Description of changes
This PR changes the
Table.columns
property to return the underlyingtuple
of names from the operation'sschema
attribute.Right now, every
.columns
access incurs the overhead of creating alist
,which remained for backwards compatibility.
I was refactoring selectors in #9917, and noticed some performance differences
that boiled down to accessing this attribute many times in the case of
selectors operating on very wide tables.
The other approach I considered was t continue returning a
list
, and speed upthe access using
functools.cached_property
.While this doesn't break the API, it seemed incredibly wasteful to store two
copies of every table's columns, especially in these slower cases where
tables have a ton of columns.
BREAKING CHANGE:
Table.columns
is now atuple
instead of alist
. Your usage of.columns
may require updating if you're depending specifically on.columns
being alist
.