Error converting DataFrame with duplicate columns to ndarray

[jpindi]

Installed latest version of pandas 0.9.0 in case this was an error
Trying to read Excel file. That part seems ok.
Originally, I was trying iteritems() for each row of the pandas dataframe, as the id_company had to be verified against a mysql database (code not included). Same/similar error message to putting it into a tuple (code is below). Error message follows.

Note there is a .reindex() but it didn't work before, either. The reindex() was kind of a hail-mary.

As a work-around, I'm probably going to simply import from my target sql and do a join. I'm concerned because of the size of the datasets.

import pandas as pd
def runNow():
    #identify sheet
    source = 'C:\Users\jlalonde\Desktop\startup_geno\startupgenome_w_id_xl_20121109.xlsx'
    xls_file = pd.ExcelFile(source)
    sd = xls_file.parse('Sheet1')
    source_u = sd.drop_duplicates(cols = 'id_company', take_last=False)
    source_r = source_u[['id_company','id_good','description', 'website','keyword', 'company_name','founded_month', 'founded_year', 'description']]
    source_i = source_r.reindex() #hail mary
    tup_r = [tuple(x) for x in source_i.values]

Traceback (most recent call last):
  File "<pyshell#10>", line 1, in <module>
    sg_sql_2.runNow()
  File "sg_sql_2.py", line 31, in runNow
    tup_r = [tuple(x) for x in source_r.values]
  File "C:\Python27\lib\site-packages\pandas\core\frame.py", line 1443, in as_matrix
    return self._data.as_matrix(columns).T
  File "C:\Python27\lib\site-packages\pandas\core\internals.py", line 723, in as_matrix
    mat = self._interleave(self.items)
  File "C:\Python27\lib\site-packages\pandas\core\internals.py", line 743, in _interleave
    indexer = items.get_indexer(block.items)
  File "C:\Python27\lib\site-packages\pandas\core\index.py", line 748, in get_indexer
    raise Exception('Reindexing only valid with uniquely valued Index '
Exception: Reindexing only valid with uniquely valued Index objects

[wesm]

Any chance you could e-mail me the excel file? wesmckinn at gmail

[jpindi]

Done! Just a note that I am using Python 2.7 and the Excel file was created using Excel 2010 on Windows. Thx.

[wesm]

The reindexing error is a red herring. I updated the title to reflect the underlying bug and I'll take a look now.

[jpindi]

You rock, Wes. Do I have to reinstall or do anything else? Just say the word...

[wesm]

Yeah you'll have to install the dev version or wait til 0.9.1 is cut this
week l. There are various workarounds you could do-- basically just have to
make the columns not have duplicate entries

Sent from my mobile device

On Nov 13, 2012, at 8:07 PM, Joseph Lalonde notifications@github.com
wrote:

You rock, Wes. Do I have to reinstall or do anything else? Just say the
word...

—
Reply to this email directly or view it on
GitHubhttps://github.com//issues/2236#issuecomment-10351223.