PERF: speed up MultiIndex.is_monotonic by 50x

[qwhelan]

The current logic for MultiIndex.is_monotonic relies on np.lexsort() on MultiIndex._values. While the result is cached, this is slow as it triggers the creation of ._values, needs to perform an O(n log(n)) sort, as well as populate the hashmap of a transient Index.

This PR significantly speeds up this check by directly operating on .codes when .levels are individually sorted. This means we can leverage libalgos.is_lexsorted() which is O(n) (but has the downside of needing int64 when MultiIndex compacts levels).

       before           after         ratio
     [5bd57f90]       [3320dded]
-      31.9±0.9ms         627±50μs     0.02  index_cached_properties.IndexCache.time_is_monotonic_decreasing('MultiIndex')
-      29.6±0.7ms         528±80μs     0.02  index_cached_properties.IndexCache.time_is_monotonic('MultiIndex')
-        29.9±1ms         524±90μs     0.02  index_cached_properties.IndexCache.time_is_monotonic_increasing('MultiIndex')

• closes Join operation takes more time.  #27099
• tests added / passed
• passes black pandas
• passes git diff upstream/master -u -- "*.py" | flake8 --diff
• whatsnew entry

[jreback]

can you add a note in Performance in 1.0 whatsnew

[jreback]

can you add a comment on what this short-circuits. how often is this hit (as compared to the following path)?

[qwhelan]

Of course - it appears to almost always get hit, as it seems levels get sorted as part of .codes creation.

[jreback]

hmm, when is it not hit?

can you simply sort if its not hit then call the same code (and remove what's below)?

[qwhelan]

Having .levels be monotonic gives us the property that monotonicity in code-space is monotonic in level-space (as the values in a level are unique). If we sort a level to obtain monotonicity, we'd have to re-encode that level.

One case is when MultiIndex(codes=..., levels=...) is directly constructed and the levels are not sorted. It could be a win to re-encode here, but I'll need to do some testing.

[jreback]

ok, not a big deal; the reason is mainly that we can remove code :-> happy to merge this and can investigate that as a followup (or here ok too). lmk.

[qwhelan]

Let's merge this for now - I created a followup issue here: #27498

[jreback]

you maybe be able to add copy=False here

[qwhelan]

Looks like that nets another ~10-20% gain - thanks!

[jreback]

lol

[jreback]

thanks @qwhelan