BUG: in Python3 MultiIndex.from_tuples cannot take "zipped" tuples #18440

Xbar · 2017-11-23T02:43:00Z

MultiIndex.from_tuples accept zipped tuples in python 3. Ensures compatibility
between 2 and 3.

closes BUG: in Python3 MultiIndex.from_tuples cannot take "zipped" tuples #18434
tests added / passed
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

codecov · 2017-11-23T08:25:47Z

Codecov Report

Merging #18440 into master will decrease coverage by 0.03%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #18440      +/-   ##
==========================================
- Coverage   91.36%   91.32%   -0.04%     
==========================================
  Files         163      163              
  Lines       49700    49764      +64     
==========================================
+ Hits        45407    45446      +39     
- Misses       4293     4318      +25

Flag	Coverage Δ
#multiple	`89.12% <100%> (-0.03%)`	⬇️
#single	`40.71% <50%> (+0.99%)`	⬆️

Impacted Files	Coverage Δ
pandas/core/indexes/multi.py	`96.44% <100%> (+0.03%)`	⬆️
pandas/io/gbq.py	`25% <0%> (-58.34%)`	⬇️
pandas/io/clipboard/__init__.py	`56.86% <0%> (-19.33%)`	⬇️
pandas/core/dtypes/missing.py	`90.24% <0%> (-0.5%)`	⬇️
pandas/core/indexes/datetimelike.py	`96.91% <0%> (-0.2%)`	⬇️
pandas/core/frame.py	`97.8% <0%> (-0.1%)`	⬇️
pandas/core/series.py	`94.76% <0%> (-0.08%)`	⬇️
pandas/core/dtypes/cast.py	`88.52% <0%> (-0.08%)`	⬇️
pandas/core/indexes/base.py	`96.4% <0%> (-0.04%)`	⬇️
pandas/core/base.py	`96.54% <0%> (-0.03%)`	⬇️
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update aaee541...7a220aa. Read the comment docs.

jreback · 2017-11-23T16:00:55Z

pandas/core/indexes/multi.py

@@ -1206,6 +1206,12 @@ def from_tuples(cls, tuples, sortorder=None, names=None):
        MultiIndex.from_product : Make a MultiIndex from cartesian product
                                  of iterables
        """
+        if not hasattr(tuples, '__len__'):


would just do

if not is_list_like(tuples): raise TypeError... elif is_iterator(tuples): tuples = list(tuples)

jreback · 2017-11-23T16:01:41Z

pandas/core/indexes/multi.py

@@ -1206,6 +1206,12 @@ def from_tuples(cls, tuples, sortorder=None, names=None):
        MultiIndex.from_product : Make a MultiIndex from cartesian product
                                  of iterables
        """
+        if not hasattr(tuples, '__len__'):
+            try:
+                iter(tuples)  # if iterable, then it's iterator or generator


needs checking in from_product and from_arrays as well.

needs tests that hit all 3 cases

Xbar · 2017-11-24T03:19:52Z

Added the test cases and fix from_tuples/from_arrays/from_product

jreback

some error context comments. otherwise lgtm. ping on green.

jreback · 2017-11-24T17:00:49Z

pandas/core/indexes/multi.py

@@ -1162,6 +1162,11 @@ def from_arrays(cls, arrays, sortorder=None, names=None):
        MultiIndex.from_product : Make a MultiIndex from cartesian product
                                  of iterables
        """
+        if not is_list_like(arrays):
+            raise TypeError("Inappropriate input for list of arrays.")


make a more user-friendly message, maybe

input must be a list-like of arrays

jreback · 2017-11-24T17:00:59Z

pandas/core/indexes/multi.py

@@ -1206,6 +1211,11 @@ def from_tuples(cls, tuples, sortorder=None, names=None):
        MultiIndex.from_product : Make a MultiIndex from cartesian product
                                  of iterables
        """
+        if not is_list_like(tuples):
+            raise TypeError('Inappropriate input for tuples.')


same (except say tuples)

jreback · 2017-11-24T17:01:04Z

pandas/core/indexes/multi.py

@@ -1260,6 +1270,11 @@ def from_product(cls, iterables, sortorder=None, names=None):
        from pandas.core.categorical import _factorize_from_iterables
        from pandas.core.reshape.util import cartesian_product

+        if not is_list_like(iterables):


jreback · 2017-11-24T17:01:16Z

pandas/tests/indexes/test_multi.py

        result = MultiIndex.from_arrays(arrays)
        assert list(result) == list(self.index)
+        # iterator as input


blank line here

jreback · 2017-11-24T17:01:20Z

pandas/tests/indexes/test_multi.py

+        # iterator as input
+        result2 = MultiIndex.from_arrays(iter(arrays))
+        assert list(result2) == list(self.index)
+        # invlide iterator input


jreback · 2017-11-24T17:01:29Z

pandas/tests/indexes/test_multi.py

+        expected = MultiIndex(levels=[[1, 3], [2, 4]],
+                              labels=[[0, 1], [0, 1]],
+                              names=['a', 'b'])
+        # input tuples


jreback

pls rebase as well. lgtm. ping on green.

jreback · 2017-11-24T19:48:51Z

pandas/tests/indexes/test_multi.py

@@ -827,6 +837,16 @@ def test_from_product(self):
        tm.assert_index_equal(result, expected)
        assert result.names == names

+        # iterator as input


can you make this a separate test (the added code), and do so for from_product, from_tuples, from_arrays as well, call it something like

test_multiindex_from_product_iterable and add the issue number to it.

jreback · 2017-11-24T19:49:07Z

pandas/core/indexes/multi.py

@@ -1206,6 +1211,11 @@ def from_tuples(cls, tuples, sortorder=None, names=None):
        MultiIndex.from_product : Make a MultiIndex from cartesian product
                                  of iterables
        """
+        if not is_list_like(tuples):
+            raise TypeError('Input must be a list /sequence of tuple-likes.')


need a space after the /

jreback · 2017-11-24T19:49:27Z

pandas/tests/indexes/test_multi.py

+                              labels=[[0, 1], [0, 1]],
+                              names=['a', 'b'])
+
+        # input tuples


same (new test)

jreback · 2017-11-24T19:49:51Z

pandas/tests/indexes/test_multi.py

+                              names=['a', 'b'])
+
+        # input tuples
+        res1 = MultiIndex.from_tuples(((1, 2), (3, 4)), names=['a', 'b'])


use result =

jreback · 2017-11-24T19:49:56Z

pandas/tests/indexes/test_multi.py

+        assert res1.equals(expected)
+
+        # input iterator for tuples
+        res2 = MultiIndex.from_tuples(zip([1, 3], [2, 4]), names=['a', 'b'])


jreback · 2017-11-24T19:50:28Z

pandas/tests/indexes/test_multi.py

+        # input iterator for tuples
+        res2 = MultiIndex.from_tuples(zip([1, 3], [2, 4]), names=['a', 'b'])
+        assert expected.names == res2.names
+        assert res2.equals(expected)


use tm.assert_index_equal(result, expected) for the checking

jreback · 2017-11-24T19:51:02Z

pandas/tests/indexes/test_multi.py

        result = MultiIndex.from_arrays(arrays)
        assert list(result) == list(self.index)

+        # iterator as input


separate test

jreback · 2017-11-24T19:51:13Z

pandas/core/indexes/multi.py

-        for lev, lab in zip(self.levels, self.labels):
+        for idx, (lev, lab) in enumerate(zip(self.levels, self.labels)):
+            na_idxs = np.where(lab == -1)[0]
+


looks like you picked up another commit.

Xbar · 2017-11-25T01:30:00Z

All build passed.

jreback · 2017-11-25T15:08:14Z

pandas/tests/indexes/test_multi.py

+
+        # iterator as input
+        result = MultiIndex.from_arrays(iter(arrays))
+        assert list(result) == list(self.index)


use tm.assert_index_equal, IOW construct the expected MI

jreback · 2017-11-25T15:08:24Z

pandas/tests/indexes/test_multi.py

+
+        # iterator as input
+        result = MultiIndex.from_product(iter([first, second]), names=names)
+        assert result.equals(expected)


use assert_index_equal

jreback · 2017-11-25T15:08:36Z

pandas/tests/indexes/test_multi.py

+
+        # input tuples
+        result = MultiIndex.from_tuples(((1, 2), (3, 4)), names=['a', 'b'])
+        assert expected.names == result.names


jreback · 2017-11-25T15:08:40Z

pandas/tests/indexes/test_multi.py

+        # input tuples
+        result = MultiIndex.from_tuples(((1, 2), (3, 4)), names=['a', 'b'])
+        assert expected.names == result.names
+        assert result.equals(expected)


jreback · 2017-11-25T15:08:45Z

pandas/tests/indexes/test_multi.py

+                              names=['a', 'b'])
+
+        result = MultiIndex.from_tuples(zip([1, 3], [2, 4]), names=['a', 'b'])
+        assert expected.names == result.names


jreback · 2017-11-25T20:57:02Z

thanks @Xbar nice patch and very responsive interactions!

Xbar · 2017-11-25T22:13:05Z

Thanks! 👍

jreback requested changes Nov 23, 2017

View reviewed changes

jreback changed the title ~~Fix issue 18434~~ BUG: in Python3 MultiIndex.from_tuples cannot take "zipped" tuples Nov 23, 2017

jreback added 2/3 Compat Reshaping Concat, Merge/Join, Stack/Unstack, Explode Error Reporting Incorrect or improved errors from pandas labels Nov 23, 2017

Xbar force-pushed the master branch from a60467d to c2bbe5c Compare November 24, 2017 03:17

jreback requested changes Nov 24, 2017

View reviewed changes

Xbar force-pushed the master branch from b084cd4 to 7f38fd5 Compare November 24, 2017 17:24

jreback requested changes Nov 24, 2017

View reviewed changes

Xbar force-pushed the master branch from 7f38fd5 to 9ceadbe Compare November 24, 2017 21:05

Make independent test cases for issue 18434

65fbc91

Xbar force-pushed the master branch from 9ceadbe to 65fbc91 Compare November 24, 2017 21:16

jreback requested changes Nov 25, 2017

View reviewed changes

use assert_index_equal for assert

7a220aa

jreback added this to the 0.22.0 milestone Nov 25, 2017

jreback approved these changes Nov 25, 2017

View reviewed changes

jreback merged commit 0bcd77e into pandas-dev:master Nov 25, 2017

simonjayhawkins mentioned this pull request Feb 8, 2019

API: more consistent error message for MultiIndex.from_arrays #25189

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BUG: in Python3 MultiIndex.from_tuples cannot take "zipped" tuples #18440

BUG: in Python3 MultiIndex.from_tuples cannot take "zipped" tuples #18440

Xbar commented Nov 23, 2017

codecov bot commented Nov 23, 2017 •

edited

Loading

jreback Nov 23, 2017

jreback Nov 23, 2017

Xbar commented Nov 24, 2017

jreback left a comment

jreback Nov 24, 2017

jreback Nov 24, 2017

jreback Nov 24, 2017

jreback Nov 24, 2017

jreback Nov 24, 2017

jreback Nov 24, 2017

jreback left a comment

jreback Nov 24, 2017

jreback Nov 24, 2017

jreback Nov 24, 2017

jreback Nov 24, 2017

jreback Nov 24, 2017

jreback Nov 24, 2017

jreback Nov 24, 2017

jreback Nov 24, 2017

Xbar commented Nov 25, 2017

jreback Nov 25, 2017

jreback Nov 25, 2017

jreback Nov 25, 2017

jreback Nov 25, 2017

jreback Nov 25, 2017

jreback commented Nov 25, 2017

Xbar commented Nov 25, 2017

BUG: in Python3 MultiIndex.from_tuples cannot take "zipped" tuples #18440

BUG: in Python3 MultiIndex.from_tuples cannot take "zipped" tuples #18440

Conversation

Xbar commented Nov 23, 2017

codecov bot commented Nov 23, 2017 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Xbar commented Nov 24, 2017

jreback left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Xbar commented Nov 25, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Nov 25, 2017

Xbar commented Nov 25, 2017

codecov bot commented Nov 23, 2017 •

edited

Loading