Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: concat with copy=False of ExtensionArray fails #20756

Closed
jorisvandenbossche opened this issue Apr 20, 2018 · 1 comment · Fixed by #30625
Closed

BUG: concat with copy=False of ExtensionArray fails #20756

jorisvandenbossche opened this issue Apr 20, 2018 · 1 comment · Fixed by #30625
Labels
ExtensionArray Extending pandas with custom dtypes or arrays. good first issue Needs Tests Unit test(s) needed to prevent regressions
Milestone

Comments

@jorisvandenbossche
Copy link
Member

In [1]: from pandas.tests.extension.decimal.array import DecimalArray, make_data

In [5]: dec_arr = DecimalArray(make_data())

In [6]: df1 = pd.DataFrame({'int1': [1, 2, 3], 'key':[0, 1, 2], 'ext1': dec_arr[:3]})

In [7]: df2 = pd.DataFrame({'int2': [1, 2, 3, 4], 'key':[0, 0, 1, 3], 'ext2': dec_arr[3:7]})

In [8]: pd.concat([df1, df1], axis=1, copy=False)
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-60-bd0d3639db5b> in <module>()
----> 1 pd.concat([df1, df1], axis=1, copy=False)

/home/joris/scipy/pandas/pandas/core/reshape/concat.py in concat(objs, axis, join, join_axes, ignore_index, keys, levels, names, verify_integrity, copy)
    211                        verify_integrity=verify_integrity,
    212                        copy=copy)
--> 213     return op.get_result()
    214 
    215 

/home/joris/scipy/pandas/pandas/core/reshape/concat.py in get_result(self)
    406             new_data = concatenate_block_managers(
    407                 mgrs_indexers, self.new_axes, concat_axis=self.axis,
--> 408                 copy=self.copy)
    409             if not self.copy:
    410                 new_data._consolidate_inplace()

/home/joris/scipy/pandas/pandas/core/internals.py in concatenate_block_managers(mgrs_indexers, axes, concat_axis, copy)
   5394     import pdb; pdb.set_trace()
   5395 
-> 5396     for placement, join_units in concat_plan:
   5397 
   5398         if len(join_units) == 1 and not join_units[0].indexers:

AttributeError: 'DecimalArray' object has no attribute 'view'

this fails because of:

if len(join_units) == 1 and not join_units[0].indexers:
b = join_units[0].block
values = b.values
if copy:
values = values.copy()
elif not copy:
values = values.view()
b = b.make_block_same_class(values, placement=placement)

so if copy=False, it takes a view of the data, which is not defined on the extension array interface.
I am not fully sure why the view is needed here.

@jorisvandenbossche jorisvandenbossche added the ExtensionArray Extending pandas with custom dtypes or arrays. label Apr 20, 2018
@jorisvandenbossche jorisvandenbossche added the Needs Tests Unit test(s) needed to prevent regressions label Dec 18, 2019
@jorisvandenbossche jorisvandenbossche added this to the Contributions Welcome milestone Dec 18, 2019
@jorisvandenbossche
Copy link
Member Author

This is working now on master. It would still be good to add a test for this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ExtensionArray Extending pandas with custom dtypes or arrays. good first issue Needs Tests Unit test(s) needed to prevent regressions
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants