BUG: DataFrame inplace where doesn't work for mixed datatype frames

[stephenwlin]

Noticed the following: DataFrame.where with inplace=True only works when DataFrame is of a single dtype (because it relies on calling np.putmask on DataFrame.values which is only a view in the single dtype case)

In [1]: import pandas as p

In [2]: import numpy as np

In [3]: df=p.DataFrame({'a': [1.0, 2.0, 3.0, 4.0], 
   ...:                 'b': [4.0, 3.0, 2.0, 1.0]}) # single dtype

In [4]: df.where(df > 2, np.nan)
Out[4]: 
    a   b
0 NaN   4
1 NaN   3
2   3 NaN
3   4 NaN

In [5]: df.where(df > 2, np.nan, inplace=True)

In [6]: df
Out[6]: 
    a   b
0 NaN   4
1 NaN   3
2   3 NaN
3   4 NaN

In [7]: df=p.DataFrame({'a': [1, 2, 3, 4],
   ...:                 'b': [4.0, 3.0, 2.0, 1.0]}) # mixed dtypes

In [8]: df.where(df > 2, np.nan) # ok
Out[8]: 
    a   b
0 NaN   4
1 NaN   3
2   3 NaN
3   4 NaN

In [9]: df.where(df > 2, np.nan, inplace=True) # not ok

In [10]: df
Out[10]: 
   a  b
0  1  4
1  2  3
2  3  2
3  4  1

[jreback]

fixed in #2708

a little bit non-trivial as you can (and in your example you did) have an upcast from integer to float, in
an in-place operation, causing the underlying blocks to change; so have to modify the manager
in place

good news is that I wrote a comprehensive test suite for where catching this (and a couple of other) issues

thanks!

In [11]: df = pd.DataFrame({
  'a': np.array([1, 2, 3, 4],dtype='int32'), 
  'b': np.array([4.0, 3.0, 2.0, 1.0], dtype = 'float64') })

In [12]: df.dtypes
Out[12]: 
a      int32
b    float64
Dtype: object

In [13]: df.where(df>2,np.nan,inplace=True)

In [14]: df.dtypes
Out[14]: 
a    float64
b    float64
Dtype: object

In [9]: df
Out[9]: 
    a   b
0 NaN   4
1 NaN   3
2   3 NaN
3   4 NaN

In [15]: df = pd.DataFrame({
  'a': np.array([1, 2, 3, 4],dtype='int64'), 
   'b': np.array([4.0, 3.0, 2.0, 1.0], dtype = 'float64') })

In [16]: df.dtypes
Out[16]: 
a      int64
b    float64
Dtype: object

In [17]: df.where(df>2,np.nan,inplace=True)

In [18]: df.dtypes
Out[18]: 
a    float64
b    float64
Dtype: object

In [9]: df
Out[9]: 
    a   b
0 NaN   4
1 NaN   3
2   3 NaN
3   4 NaN

[stephenwlin]

oh, ok, that's great! i'll close this then.