s390x (big endian, 64bit) + python 3.3.4 make pandas very unhappy

[yarikoptic]

http://www.onerussian.com/tmp/build-pandas-0.13.1-s390x.log

FAILED (SKIP=350, errors=87, failures=47)

Just FYI and may be anything there hints you on some possible issues

[jreback]

it all looks like the same error

this is the constructor from a dict which is used everywhere

maybe u can have a test debug at that point and print out all the variables

it's failing in a list comprehension. that is standard python - shouldn't fail there

[yarikoptic]

On Mon, 10 Feb 2014, jreback wrote:

it all looks like the same error

this is the constructor from a dict which is used everywhere

maybe u can have a test debug at that point and print out all the
variables

it's failing in a list comprehension. that is standard python - shouldn't
fail there

I guess you mean the first one in the list of many errors:

yoh@zelenka:~/pandas$ grep Error build.log | sort | uniq -c | sort -rn | head
     58 KeyError: 1
     17     raise KeyError('no item named %s' % com.pprint_thing(item))
      8 AssertionError: attr is not equal [dtype]: dtype('O') != dtype('float64')
      6 KeyError: 45
      4 AssertionError: attr is not equal [dtype]: dtype('float64') != dtype('O')
      3 KeyError: 'no item named (2000, 1, 3)'
      3 KeyError: 'no item named (0, 1)'
      3 AssertionError: nan != '\xc1 k\xf6ldum klaka (Cold Fever) (1994)'
      3 AssertionError: First object is null, second isn't: nan != 2
      3 AssertionError: attr is not equal [dtype]: dtype('O') != dtype('int64')
...

ok -- will check for KeyError fist... smells like numpy issue (they must
love s390 by now) and ATM I forgotten if/how it is different from
the similar one we troubleshooted with Julian(CCed) before:

(Pdb) data.keys()
dict_keys([0, 1, 2, 3, 4])
(Pdb) k
1
(Pdb) data[k]
*** KeyError: 1
(Pdb) data[1]
2000-01-03   -0.107642
2000-01-04   -0.327630
2000-01-05   -0.539336
2000-01-06   -1.556103
2000-01-07    0.170068
2000-01-10   -0.033355
2000-01-11    0.053571
2000-01-12   -0.238378
2000-01-13    0.001964
2000-01-14   -0.128373
2000-01-17   -1.241692
2000-01-18   -2.476735
2000-01-19   -0.950525
2000-01-20    0.971429
2000-01-21   -0.653595
2000-01-24    0.434982
2000-01-25    0.689321
2000-01-26    0.908540
2000-01-27   -0.120851
2000-01-28    0.297481
2000-01-31    0.151511
2000-02-01   -1.227303
2000-02-02    1.328172
2000-02-03   -1.391838
2000-02-04   -0.686879
2000-02-07   -1.348318
2000-02-08    0.361292
2000-02-09   -0.562358
2000-02-10    0.274780
2000-02-11    0.109150
Name: B, dtype: float64
(Pdb) type(k)
<class 'numpy.int64'>

Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
WWW: http://www.linkedin.com/in/yarik

[yarikoptic]

might be the hashing bug fixed in numpy 1.8 or 1.7.2, you can use the
packages from experimental to check.

[jreback]

have you tested s390x before with any pandas that has completely worked? This is a newer supported platform in debian land, yes?

[yarikoptic]

I think that while testing using numpy 1.8.0 I have managed to lock it
up similarly as it locked up during original build by debian build bots:

test_non_unique_loc_memory_error (pandas.tests.test_indexing.TestIndexing) ...

never returns... happens in

#0 0x0000000080029330 in ?? ()
#1 0x000000008002ad00 in ?? ()
#2 0x000000008002b8e0 in ?? ()
#3 0x0000000080038c32 in ?? ()
#4 0x00000000801cda20 in PyObject_Call ()
#5 0x0000020001ff06f8 in __pyx_pf_6pandas_5index_11IndexEngine_16get_indexer_non_unique (__pyx_v_self=0x200073cc150, __pyx_v_targets=0xad553010)
at pandas/index.c:5416
#6 0x00000000800a3b94 in PyEval_EvalFrameEx ()
#7 0x00000000800a5096 in PyEval_EvalCodeEx ()
#8 0x00000000800a3858 in PyEval_EvalFrameEx ()
(gdb) c
Continuing.
^C
Program received signal SIGINT, Interrupt.
0x00000000800206a2 in PyObject_RichCompareBool ()

btw -- cython is whining a bit

cythoning pandas/index.pyx to pandas/index.c
warning: pandas/src/numpy.pxd:873:17: Non-trivial type declarators in shared declaration (e.g. mix of pointers and values). Each pointer declaration should be on its own line.
warning: pandas/src/numpy.pxd:873:24: Non-trivial type declarators in shared declaration (e.g. mix of pointers and values). Each pointer declaration should be on its own line.

in

(experimental_s390x-dchroot)yoh@zelenka:~/pandas/pandas-0.13.1$ nl -ba pandas/src/numpy.pxd | grep -1 873
872 int check_return
873 char *name, *types
874 char *doc

so might be worth obeying and placing them on sep lines

On Tue, 11 Feb 2014, jreback wrote:

have you tested s390x before with any pandas that has completely worked?
This is a newer supported platform in debian land, yes?

—
Reply to this email directly or [1]view it on GitHub.

References

Visible links

1. s390x (big endian, 64bit) + python 3.3.4 make pandas very unhappy #6320 (comment)

Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
WWW: http://www.linkedin.com/in/yarik

[jreback]

closing @yarikoptic pls open a new issue if 0.14.0 RC experiences issues