DEPR: Panel deprecated

[jreback]

closes #13563

going to be on top of #15677

Partially addresses #14565 (comment)

Note this is currently a FutureWarning mainly to fail the tests if I missed anything. Intention is to change to a DeprecationWarning.

[jreback]

@wesm @jorisvandenbossche @TomAugspurger @sinhrks @chris-b1 @cpcloud @shoyer

[codecov-io]

Codecov Report

Merging #15601 into master will decrease coverage by <.01%.
The diff coverage is 94.28%.

@@            Coverage Diff             @@
##           master   #15601      +/-   ##
==========================================
- Coverage   90.99%   90.99%   -0.01%     
==========================================
  Files         145      145              
  Lines       49521    49537      +16     
==========================================
+ Hits        45061    45074      +13     
- Misses       4460     4463       +3

Flag	Coverage Δ
#multiple	88.75% <94.28%> (-0.01%)	⬇️
#single	40.58% <17.14%> (-0.04%)	⬇️

Impacted Files	Coverage Δ
pandas/util/testing.py	80.88% <100%> (+0.03%)	⬆️
pandas/core/panel.py	97.23% <100%> (ø)	⬆️
pandas/core/window.py	96.22% <100%> (+0.04%)	⬆️
pandas/indexes/multi.py	96.7% <100%> (-0.01%)	⬇️
pandas/indexes/frozen.py	91.17% <66.66%> (-2.38%)	⬇️
pandas/core/indexing.py	94.01% <0%> (-0.1%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d9e00d2...04104a7. Read the comment docs.

[jreback]

getting rid of a deprecation warning that was being trapped :>

[jreback]

same

[TomAugspurger]

I'm a bit behind on the discussion here, but have we talked about how we'll handle things like rolling corr and cov?

In [17]: df = pd.DataFrame(np.random.randn(100, 2))

In [18]: df.rolling(12).corr()
Out[18]:
<class 'pandas.core.panel.Panel'>
Dimensions: 100 (items) x 2 (major_axis) x 2 (minor_axis)
Items axis: 0 to 99
Major_axis axis: 0 to 1
Minor_axis axis: 0 to 1

[jorisvandenbossche]

That's a good point, I don't think we already discussed that.
One option would be to follow the recommendation given in the deprecation warning and use a MultiIndex (with some deprecation strategy in rolling corr itself of course). Directly calling to_frame on the result is not the desired result, but if we swap some axes:

In [18]: p = df.rolling(12).corr()

In [21]: p.swapaxes(0,1).to_frame(filter_observations=False)
Out[21]: 
                    0         1
major minor                    
0     0           NaN       NaN
      1           NaN       NaN
1     0           NaN       NaN
      1           NaN       NaN
2     0           NaN       NaN
      1           NaN       NaN
3     0           NaN       NaN
      1           NaN       NaN
4     0           NaN       NaN
      1           NaN       NaN
...               ...       ...
96    0      1.000000 -0.437443
      1     -0.437443  1.000000
97    0      1.000000 -0.481589
      1     -0.481589  1.000000
98    0      1.000000 -0.094523
      1     -0.094523  1.000000
99    0      1.000000 -0.145248
      1     -0.145248  1.000000

[200 rows x 2 columns]

[jreback]

well I would simply change this to return a MI DataFrame as @jorisvandenbossche shows. This solves the problem and just fixes it.

We could also just show a FutureWarning (a specific one that this is going to change to a MI DataFrame in the next version). But that just defers the change itself, and getting warning of the change really doesn't help your code in this case as you have a different object being returned.

any objections to simply changing it? (I will do this as a separate PR). Further I think we should actually name these axes (self, other), maybe (arg1, ``arg2)?

[jreback]

@jorisvandenbossche all pushed

Here is the whatsnew example

In [1]:    df = pd.DataFrame(np.random.rand(100, 2),
   ...:                      columns=pd.Index(['A', 'B'], name='bar'),
   ...:                      index=pd.date_range('20160101',
   ...:                                          periods=100, freq='D', name='foo'))

In [2]: df.tail()
Out[2]: 
bar                A         B
foo                           
2016-04-05  0.603159  0.694132
2016-04-06  0.466162  0.577965
2016-04-07  0.970528  0.403820
2016-04-08  0.837619  0.583224
2016-04-09  0.039695  0.029523

In [3]: df.rolling(12).corr().tail()
Out[3]: 
bar                    A         B
foo        bar                    
2016-04-07 B    0.161726  1.000000
2016-04-08 A    1.000000  0.222413
           B    0.222413  1.000000
2016-04-09 A    1.000000  0.380243
           B    0.380243  1.000000

[shoyer]

I haven't looked at the code in details, but the docs look great here -- thanks @jreback !

[jreback]

and bombs away. sad day :<

[wesm]

Perhaps we can think about panels again as a way to manage a collection of DataFrame objects in C++ once pandas 2.0 hits the shelves

[jreback]

yep. I think was of the biggest criticisms was the indexing (how the axes were named), and the fact that indexing was not the same as numpy.