RLS: 0.25.0

[jreback]

Tracking issue for 0.25.0

[jreback]

@pandas-dev/pandas-core

as we briefly discussed before, we need to be pretty circumspect about merging even to master changes that are not backportable to 0.24.x, prob for a little while. Small changes ok, but sweeping reorgs just make backporting much more difficult (as may need to go thru potentially a lot of effort to backport). So am going to suggest that we hold off on 'bigger' things until 0.24.1 is out.

• this means don't remove any PY2 removal stuff quite yet.
• we may need to keep merging master on soon to be merged PRs, so the backlog may grow.
• bug fixes, by all means, sometimes this may mean narrowing the scope of a bug fix though (meaning that if the 'right' way to do it involves a fairly big reorg).

[jbrockmendel]

Do we have any major goals for 0.25.0?

[jreback]

• deprecate remaining items on the list
• remove PY2

[h-vetinari]

I think one of the key points is still to decide on a policy for dealing with breaking changes, both for 0.25/1.0 and after.

There's still a bunch of inconsistencies that we'd be locking in for 1.0, and even more so if we'd be doing actual SemVer afterwards - changing a None to a np.nan somewhere would then already count as breaking).

• groupby: API/DOC: clean up DataFrame.groupby.apply #22545 (+ Inconsistent groupby-apply output shape and random values returned. #20420, BUG: index of group not returned correctly in groupby.apply #22541, API: groupby aggregation with apply does not drop groupby-column #22542, BUG: weird behaviour for returning group in groupby.apply #22546, ...)
• consistent IO-API: API: formalize the pandas IO API #15862, API: Unify compression-kwarg for IO-methods #21640, maybe CLN: spin msgpack support to [new] pandas-msgpack #15841
• boolean Series: Mismatch in Series/DataFrame boolean ops behavior #22724, NaN values impact binary or operations asymmetrically #6528, "Or" does not have commutative property (possibly because it catches exceptions silently?) #13806

This list is obviously not exhaustive. Pandas is certainly very mature by now, but IMO not consistent enough to do SemVer (otherwise every release would be major).

Maybe numpy-style rolling deprecations (as we've been doing as well) would be less restrictive to actually fixing some of these issues (it's worth noting that on the numpy side, it will still take monumental determination and effort to actually fix some very longstanding issues for an eventual 2.0)

There's much smaller fish I'd like to fry as well, but all of which are breaking in some way.

• inferring types for .str-accessor; moving bytes to separate accessor; Disable .str-accessor for byte data #23011 API: Series.str-accessor infers dtype (and Index.str does not raise on all-NA) #23167+ several fixes that are blocked on that refactor
• fixing maybe_promote and maybe_upcast_putmask BUG/Internals: maybe_upcast_putmask #23823 BUG/Internals: maybe_promote #23833 to then fix .update etc. API/BUG: unnecessary upcasting for df.update #23606
• adding the long-open inverses to unique API: provide a better way of doing np.unique(return_inverses=True) #4087, API/ENH: overhaul/unify/improve .unique #22824, ENH: adding .unique() to DF (or return_inverse for duplicated) #21357, which needs that Series.unique stays a Series (which would also fit very well with the current drive from 0.24 that one needs .to_numpy() to get an ndarray)

TL;DR: most bigger PRs are breaking from the point of SemVer. A policy of zero breaking changes until 1.0 will probably mean little development.

[stonebig]

What is the likely pandas-0.25.0 release date ?

[jorisvandenbossche]

@stonebig we didn't really discuss in detail recently, but I think we still target to release somewhere in June (but no guarantees!)

[TomAugspurger]

Are we on track for cutting an RC today? (my gut says "not quite").

I'm fixing up a few CI issues across pandas & dask right now. Will do a run through outstanding issues after that (I know I have the __array_ufunc__ PR at least).

[jreback]

yeah let’s get ci fixed and merge what we can
tomorrow

[jreback]

@jorisvandenbossche @TomAugspurger I am super busy today / tomorrow, so don't block on me for things and merge where you guys think appropriate. if you want me to specifically review something, pls ping.

[jorisvandenbossche]

@jreback I would like to have your go for #17753

[TomAugspurger]

@pandas-dev/pandas-core I think #27095 is the last one for the RC.

I'll try to tag 0.25.0rc0 tonight (~5 hours from now).

LMK if there are any others that should go in.

[jorisvandenbossche]

If we do #26043, I think should include that in the RC as well (but seems green apart from some code check)

[jreback]

I just pushed a patch for #26043 - hopefullly should be green soon.

[WillAyd]

If #25427 is mergeable would be good to get in before release

[jschendel]

I just opened #27221; I've marked it as 0.25.0 but I'm not dead set on getting it into 0.25.0. No need to hold up the RC for it.

I have a couple other small Interval related things in mind that I want to work on in the next day, but same deal with not being dead set on 0.25.0. They should be relatively self-contained, so it might be fine to merge them as part of 0.25.0 after the RC but before the official release? Otherwise perfectly fine to push them to the next release.

[jbrockmendel]

Is it too late to remove _IXIndexer before for 0.25.0? I'm finding some ugly code in internals is only reached via ix

[TomAugspurger]

Feels a bit late. I'd like to tag the RC soon (next hour if possible).

[jorisvandenbossche]

Is it too late to remove _IXIndexer before for 0.25.0? I'm finding some ugly code in internals is only reached via ix

We only changed the warning to FutureWarning this release, so it in any case only to remove for 1.0 I think.

@jschendel for the interval ones, if they are new features that don't really touch existing behaviours, we can maybe merge some after the RC

[jorisvandenbossche]

OK, black went in, so all ready to tag I think (I am offline now)

[TomAugspurger]

@pandas-dev/pandas-core tagged 0.25.0rc0.

Binary packages are building at conda-forge/pandas-feedstock#64 and MacPython/pandas-wheels#54.

Building the docs. Will push them tomorrow morning and send out the announcement.

[jorisvandenbossche]

There is a lot of activity (which is great), but since we put a RC out, I think we should be a bit careful with merging a lot of refactoring / cleaning PRs (they always have a small risk of breaking something that was not covered) or further PRs removing things (I just commented on #27377).

Instead of holding up some PRs, we could also already branch 0.25.x and specifically pick PRs to backport? (with the bot that goes rather smooth).

[jreback]

#27288 needs to be fixed
and #27267 is ready otherwise we should just release

the rc don’t have much testing compared with an actual release

[bashtage]

Any idea about the timeframe for 0.25? statsmodels needs a point release I think for 0.25 and so it would be good to have it ready to go when pandas releases.

[TomAugspurger]

@bashtage will let you know later today.

---

@jreback @jorisvandenbossche I'm back online today, getting caught up. I recall seeing a message about an API breaking change being merged in the RC, so we'll need to revert that or have a second RC? I'll post when I find what I'm talking about.

[jreback]

I recall seeing a message about an API breaking change being merged in the RC

nothing significant has been merged, so I think the release is pretty good (ex my comments above).

[WillAyd]

Maybe want to revert #27252 - I can try to get to that in next day or so but if someone else wants to tackle in the meantime wouldn't mind

[jreback]

@TomAugspurger ok just a couple issues that are on for 0.25 (the viz things), other than that lgtm.

[WillAyd]

As of now there are two plotting issues still tagged for 0.25 #27152 and #26747 - are these still valid? I don't see any open PRs referencing them so OK to remove from the milestone?

There's one open PR #27384 but looks to be near completion; any objection to calling it a release after that gets merged?

[TomAugspurger]

I merged #27384

@WillAyd do you want to make a PR fixing the date on the 0.25 release note?

[TomAugspurger]

I think the plotting issues are OK to push (and #27152 can maybe be closed. Need to check).

[WillAyd]

Released!

[jreback]

thanks for the release @WillAyd !