Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve speed and memory usage with simple integer indexes #2420

Closed
wesm opened this issue Dec 3, 2012 · 1 comment
Closed

Improve speed and memory usage with simple integer indexes #2420

wesm opened this issue Dec 3, 2012 · 1 comment
Labels
Enhancement Performance Memory or execution speed performance
Milestone

Comments

@wesm
Copy link
Member

wesm commented Dec 3, 2012

http://stackoverflow.com/questions/13692418/does-pandas-cache-values-on-ix-call

I believe this will be mostly remedied when the equivalent of the range-index branch is merged.

import pandas as pd
import numpy as np
import datetime as dt
import matplotlib.cbook as mc
import inspect

print 'pandas', pd.__version__

li_list = []
for i in range(50):
    li_list.append(pd.DataFrame(data=np.random.randn(50, 17000)))

def first_element():
    a = 0
    for i in range(50):
        b = li_list[0] #Only access first element
        for j in b.columns:
            a += b.ix[i, j]

def all_in_list():
    a = 0
    for i in range(50):
        b = li_list[i] #Access all in list
        for j in b.columns:
            a += b.ix[i, j]
            print(inspect.getabsfile(b.__class__))
            break

if __name__ == '__main__':
    # first_element()
    all_in_list()
    print(mc.report_memory())
@jreback
Copy link
Contributor

jreback commented Sep 21, 2016

closed by #11892

@jreback jreback closed this as completed Sep 21, 2016
@jorisvandenbossche jorisvandenbossche modified the milestones: 0.18.0, Someday Sep 21, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement Performance Memory or execution speed performance
Projects
None yet
Development

No branches or pull requests

3 participants