Inconsistent behaviour of .ix between list and scalar key with (missing) ints #20753

toobaz · 2018-04-20T05:31:59Z

Code Sample, a copy-pastable example if possible

In [2]: pd.Series(range(3)).ix[5]
/usr/bin/ipython3:1: DeprecationWarning: 
.ix is deprecated. Please use
.loc for label based indexing or
.iloc for positional indexing

See the documentation here:
http://pandas.pydata.org/pandas-docs/stable/indexing.html#ix-indexer-is-deprecated
  #! /bin/sh
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
/home/nobackup/repo/pandas/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   3017             try:
-> 3018                 return self._engine.get_loc(key)
   3019             except KeyError:

/home/nobackup/repo/pandas/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc (pandas/_libs/index.c:5729)()
    138 
--> 139     cpdef get_loc(self, object val):
    140         if is_definitely_invalid_key(val):

/home/nobackup/repo/pandas/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc (pandas/_libs/index.c:5575)()
    160         try:
--> 161             return self.mapping.get_item(val)
    162         except (TypeError, ValueError):

/home/nobackup/repo/pandas/pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.Int64HashTable.get_item (pandas/_libs/hashtable.c:15398)()
    957 
--> 958     cpdef get_item(self, int64_t val):
    959         cdef khiter_t k

/home/nobackup/repo/pandas/pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.Int64HashTable.get_item (pandas/_libs/hashtable.c:15342)()
    963         else:
--> 964             raise KeyError(val)
    965 

KeyError: 5

During handling of the above exception, another exception occurred:

KeyError                                  Traceback (most recent call last)
/home/nobackup/repo/pandas/pandas/core/indexing.py in _get_label(self, label, axis)
    131             try:
--> 132                 return self.obj._xs(label, axis=axis)
    133             except:

/home/nobackup/repo/pandas/pandas/core/generic.py in xs(self, key, axis, level, drop_level)
   2991         else:
-> 2992             loc = self.index.get_loc(key)
   2993 

/home/nobackup/repo/pandas/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   3019             except KeyError:
-> 3020                 return self._engine.get_loc(self._maybe_cast_indexer(key))
   3021 

/home/nobackup/repo/pandas/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc (pandas/_libs/index.c:5729)()
    138 
--> 139     cpdef get_loc(self, object val):
    140         if is_definitely_invalid_key(val):

/home/nobackup/repo/pandas/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc (pandas/_libs/index.c:5575)()
    160         try:
--> 161             return self.mapping.get_item(val)
    162         except (TypeError, ValueError):

/home/nobackup/repo/pandas/pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.Int64HashTable.get_item (pandas/_libs/hashtable.c:15398)()
    957 
--> 958     cpdef get_item(self, int64_t val):
    959         cdef khiter_t k

/home/nobackup/repo/pandas/pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.Int64HashTable.get_item (pandas/_libs/hashtable.c:15342)()
    963         else:
--> 964             raise KeyError(val)
    965 

KeyError: 5

During handling of the above exception, another exception occurred:

KeyError                                  Traceback (most recent call last)
<ipython-input-2-bba00cd402c6> in <module>()
----> 1 pd.Series(range(3)).ix[5]

/home/nobackup/repo/pandas/pandas/core/indexing.py in __getitem__(self, key)
    118 
    119             key = com._apply_if_callable(key, self.obj)
--> 120             return self._getitem_axis(key, axis=axis)
    121 
    122     def _get_label(self, label, axis=None):

/home/nobackup/repo/pandas/pandas/core/indexing.py in _getitem_axis(self, key, axis)
   1104                     return self._get_loc(key, axis=axis)
   1105 
-> 1106             return self._get_label(key, axis=axis)
   1107 
   1108     def _getitem_iterable(self, key, axis=None):

/home/nobackup/repo/pandas/pandas/core/indexing.py in _get_label(self, label, axis)
    132                 return self.obj._xs(label, axis=axis)
    133             except:
--> 134                 return self.obj[label]
    135         elif isinstance(label, tuple) and isinstance(label[axis], slice):
    136             raise IndexingError('no slices here, handle elsewhere')

/home/nobackup/repo/pandas/pandas/core/series.py in __getitem__(self, key)
    753         key = com._apply_if_callable(key, self)
    754         try:
--> 755             result = self.index.get_value(self, key)
    756 
    757             if not is_scalar(result):

/home/nobackup/repo/pandas/pandas/core/indexes/base.py in get_value(self, series, key)
   3051         try:
   3052             return self._engine.get_value(s, k,
-> 3053                                           tz=getattr(series.dtype, 'tz', None))
   3054         except KeyError as e1:
   3055             if len(self) > 0 and self.inferred_type in ['integer', 'boolean']:

/home/nobackup/repo/pandas/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_value (pandas/_libs/index.c:4856)()
    103         return val in self.mapping
    104 
--> 105     cpdef get_value(self, ndarray arr, object key, object tz=None):
    106         """
    107         arr : 1-dimensional ndarray

/home/nobackup/repo/pandas/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_value (pandas/_libs/index.c:4539)()
    111             void* data_ptr
    112 
--> 113         loc = self.get_loc(key)
    114         if PySlice_Check(loc) or cnp.PyArray_Check(loc):
    115             return arr[loc]

/home/nobackup/repo/pandas/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc (pandas/_libs/index.c:5575)()
    159 
    160         try:
--> 161             return self.mapping.get_item(val)
    162         except (TypeError, ValueError):
    163             raise KeyError(val)

/home/nobackup/repo/pandas/pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.Int64HashTable.get_item (pandas/_libs/hashtable.c:15398)()
    956                                        sizeof(uint32_t)) # flags
    957 
--> 958     cpdef get_item(self, int64_t val):
    959         cdef khiter_t k
    960         k = kh_get_int64(self.table, val)

/home/nobackup/repo/pandas/pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.Int64HashTable.get_item (pandas/_libs/hashtable.c:15342)()
    962             return self.table.vals[k]
    963         else:
--> 964             raise KeyError(val)
    965 
    966     cpdef set_item(self, int64_t key, Py_ssize_t val):

KeyError: 5

In [3]: pd.Series(range(3)).ix[[5]]
/usr/bin/ipython3:1: DeprecationWarning: 
.ix is deprecated. Please use
.loc for label based indexing or
.iloc for positional indexing

See the documentation here:
http://pandas.pydata.org/pandas-docs/stable/indexing.html#ix-indexer-is-deprecated
  #! /bin/sh
Out[3]: 
5   NaN
dtype: float64

Problem description

Already fixed, opening the issue just for xref from the test/whatsnew.

Expected Output

An error

Output of `pd.show_versions()`

INSTALLED VERSIONS

commit: d04b746
python: 3.5.3.final.0
python-bits: 64
OS: Linux
OS-release: 4.9.0-6-amd64
machine: x86_64
processor:
byteorder: little
LC_ALL: None
LANG: it_IT.UTF-8
LOCALE: it_IT.UTF-8

pandas: 0.23.0.dev0+754.gd04b7464d
pytest: 3.5.0
pip: 9.0.1
setuptools: 39.0.1
Cython: 0.25.2
numpy: 1.14.1
scipy: 0.19.0
pyarrow: None
xarray: None
IPython: 6.2.1
sphinx: 1.5.6
patsy: 0.5.0
dateutil: 2.7.0
pytz: 2017.2
blosc: None
bottleneck: 1.2.0dev
tables: 3.3.0
numexpr: 2.6.1
feather: 0.3.1
matplotlib: 2.0.0
openpyxl: 2.3.0
xlrd: 1.0.0
xlwt: 1.3.0
xlsxwriter: 0.9.6
lxml: 4.1.1
bs4: 4.5.3
html5lib: 0.999999999
sqlalchemy: 1.0.15
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: 0.2.1

The text was updated successfully, but these errors were encountered:

jorisvandenbossche · 2018-04-20T07:27:55Z

This already is like that for a long time (with which I don't want to say that it is clearly an inconsistency with .loc), and given that .ix is deprecated, not sure if it is worth to fix it.

jorisvandenbossche · 2018-04-20T07:30:15Z

However, getitem has the same problem, and this is probably something more worth to fix:

In [15]: s = pd.Series(range(3))

In [16]: s[[5]]
Out[16]: 
5   NaN
dtype: float64

…so for MultiIndex closes pandas-dev#17758 closes pandas-dev#20748 closes pandas-dev#20753

…20770) * API: emit warning to raise KeyError in the future for missing keys also for MultiIndex closes #17758 closes #20748 closes #20753

jorisvandenbossche added Bug Indexing Related to indexing on series/frames, not to indexes themselves labels Apr 20, 2018

toobaz added a commit to toobaz/pandas that referenced this issue Apr 20, 2018

API: emit warning to raise KeyError in the future for missing keys al…

2d44804

…so for MultiIndex closes pandas-dev#17758 closes pandas-dev#20748 closes pandas-dev#20753

toobaz mentioned this issue Apr 20, 2018

Emit warning for missing labels in Multiindex.loc[[...]] (and more) #20770

Merged

6 tasks

toobaz added a commit to toobaz/pandas that referenced this issue Apr 22, 2018

API: emit warning to raise KeyError in the future for missing keys al…

171600d

…so for MultiIndex closes pandas-dev#17758 closes pandas-dev#20748 closes pandas-dev#20753

toobaz added a commit to toobaz/pandas that referenced this issue Apr 22, 2018

API: emit warning to raise KeyError in the future for missing keys al…

9ec8270

…so for MultiIndex closes pandas-dev#17758 closes pandas-dev#20748 closes pandas-dev#20753

jreback added this to the 0.23.0 milestone May 1, 2018

TomAugspurger closed this as completed in #20770 May 1, 2018

TomAugspurger pushed a commit that referenced this issue May 1, 2018

Emit warning for missing labels in Multiindex.loc[[...]] (and more) (#…

901fc64

…20770) * API: emit warning to raise KeyError in the future for missing keys also for MultiIndex closes #17758 closes #20748 closes #20753

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Inconsistent behaviour of .ix between list and scalar key with (missing) ints #20753

Inconsistent behaviour of .ix between list and scalar key with (missing) ints #20753

toobaz commented Apr 20, 2018

INSTALLED VERSIONS

jorisvandenbossche commented Apr 20, 2018

jorisvandenbossche commented Apr 20, 2018

Inconsistent behaviour of .ix between list and scalar key with (missing) ints #20753

Inconsistent behaviour of .ix between list and scalar key with (missing) ints #20753

Comments

toobaz commented Apr 20, 2018

Code Sample, a copy-pastable example if possible

Problem description

Expected Output

Output of pd.show_versions()

INSTALLED VERSIONS

jorisvandenbossche commented Apr 20, 2018

jorisvandenbossche commented Apr 20, 2018

Output of `pd.show_versions()`