Moving pandasjson back into mainline pandas

[wesm]

This extension works perfectly fine on Python 2.x and all platforms to my knowledge except win32 on MinGW (it's fine with VC2008+ AFAIK). It might have problems on some of @yarikoptic's esoteric Debian platforms, we should identify which ones those are and disable the extension in the setup.py until we have a chance to sort out what's wrong in the C code. I don't think that's a bad compromise for now in the interest of having a usable extension for a lot of folks.

[wesm]

also cc @Komnomnomnom

[Komnomnomnom]

Ok, so iirc the problems on the esoteric platforms were solved, at least on SPARC 32+64, if memory serves there was some memory alignment and buffer overflow problems and the fixes were pushed upstream to ujson.

As for windows, that was indeed a bit of a nightmare, the problems that occurred were intermittent segmentation faults which were hard to reproduce and pin down, and I neither had the windows tools or experience to debug it thoroughly. Here's what I concluded at the time (EPD = Enthought Python Distribution):

With an independent version of mingw -> no segmentation faults. Compiled a python debug version on win32 and compiled numpy and pandas using EPD's mingw -> no segmentation faults. From the little bit of a stack trace I've been able to get it appears that the problem occurs when ujson calls numpy C-API functions (take that with a healthy dose of salt however as the stack trace also looked pretty corrupted.)

I think the problem occurred when mixing VS compiled ujson and mingw compiled numpy, but it will take more investigation to know for sure. Ideally it would be great if someone with a proper windows setup could take a look, but maybe having it in pandas but disabled might motivate some brave soul...

On another note I think there have been some minor fixes and enhancements to ujson since it was last merged with pandasjson. Once / if pandasjson is re-included in mainline pandas I'll see about merging them in and send a pull request through?

BTW the extension should work fine in Python 3.x too, or are there some issues I'm not aware of?

[wesm]

Yes I think it should be working with python 3, I need to run the CI tests (I didn't have travis enabled when I opened the PR). If you could merge in the upstream changes sometime that would be great

[hayd]

@wesm travis is broken :(

[jreback]

@wesm @Komnomnomnom @hayd want to give #3804 a test run?

[jreback]

closed via #3804