Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MultiIndex is_monotonic_decreasing is incorrect #16554

Closed
TomAugspurger opened this issue May 31, 2017 · 4 comments · Fixed by #17455
Closed

MultiIndex is_monotonic_decreasing is incorrect #16554

TomAugspurger opened this issue May 31, 2017 · 4 comments · Fixed by #17455
Labels
Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff MultiIndex
Milestone

Comments

@TomAugspurger
Copy link
Contributor

Or just isn't implemented properly yet?
This MultiIndex is identical on the second level, so its is_monotonic_decreasing should be the same as get_level_values(0).is_monotonic_decreasing (True)

In [27]: idx = pd.MultiIndex([['baz', 'bar'], ['a', 'b']], labels=[[0, 1], [0, 0]])

In [28]: idx
Out[28]:
MultiIndex(levels=[['baz', 'bar'], ['a', 'b']],
           labels=[[0, 1], [0, 0]])

In [29]: idx.is_monotonic_decreasing
Out[29]: False

In [30]: idx.get_level_values(0).is_monotonic_decreasing
Out[30]: True

They should both return True.

@TomAugspurger TomAugspurger added Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff MultiIndex labels May 31, 2017
@TomAugspurger TomAugspurger added this to the Next Major Release milestone May 31, 2017
@jreback
Copy link
Contributor

jreback commented May 31, 2017

only implemented is_monotonic_increasing. It needs the same impl. (but its not used anywhere ATM).

@Tafkas
Copy link
Contributor

Tafkas commented Jun 1, 2017

is_monotonic_increasing is based on numpy.lexsortwhich seems not to support ordering in descending order.

@TomAugspurger
Copy link
Contributor Author

Can we define is_monotonic_decreasing in terms of increasing? Something like

def is_monotonic_decreasing(self):
    return not self.is_strictly_monotonic_increasing or idx.nunique() == 1

not sure if that will work. Looks like nunique may not be implemented on MI.

@jschendel
Copy link
Member

I don't think that will work in general. You can't get monotonic decreasing from monotonic increasing in all scenarios, unless there's an assumption that the data is monotonic/sorted in some regard. For example, a MultiIndex generated by [(3, 3), (1, 1), (2, 2)] is not monotonic decreasing, but will return True.

I think you can implement this similar to is_monotonic_increasing. It's true that numpy.lexsort only seems to support ordering in descending order, but you can get around this. My initial thought was to just reverse the output of numpy.lexsort, but that messes up the order if you have non-unique indices, e.g. [(3, 3), (2, 2), (2, 2), (1, 1)]. The fix would be to add a fake unique decreasing level to the input of numpy.lexsort to force uniqueness. Looking at the source code for is_monotonic_increasing, I think you'd just need to make changes along the lines of values = [np.arange(len(idx) - 1, -1, -1)] + [existing comprehension] followed by sort_order = np.lexsort(values)[::-1]. And in the except clause pd.Index(idx.values).is_monotonic_decreasing.

@jreback jreback modified the milestones: 0.21.0, Next Major Release Sep 7, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff MultiIndex
Projects
None yet
4 participants