Add Python 3.7.0b4

[lelit]

This is just to trigger a build and see how it goes.

[lelit]

Uhm, this confuses me:

1. why does the build completes on TravisCI, and does not on my dev machine?
2. why it says "successful check", despite this error?

[njsmith]

Oh ick, yeah, that build totally failed, but Travis is reporting it as successful, which is worrisome. The problem is that build.sh doesn't check for failures. We need to add a set -ex or something to that top of that. (CC @trishankatdatadog)

Regarding the failure itself, it looks like we're failing to build the ctypes module for some reason, though I don't know why off the top of my head. To debug this, you might want to try commenting out the >/dev/null bits in the do_cpython_build function in docker/build_scripts/build_utils.sh – that should at least let you see any messages printed during the build. (We have to keep this disabled in the real version, b/c TravisCI blows up if you print too much output during a build.)

[lelit]

I'm trying to understand why I'm not able to run the build on my local machine:

$ uname -a
Linux nautilus 4.16.0-1-amd64 #1 SMP Debian 4.16.5-1 (2018-04-29) x86_64 GNU/Linux

The exit status 139 seems related to a SEGV error, and indeed I'm not able to execute even a simple bash inside a centos:5.11 container:

$ docker run --rm -ti centos:5.11 /bin/bash
$

while I would expect something different, like:

$ docker run --rm -ti centos /bin/bash
[root@a6b8f02921be /]#

Will keep trying...

[lelit]

No way to get it working on my Debian sid 😭 but at least on macOS it seems working, going to dig there.

[lelit]

AFAICT, it fails building the ctypes extension because the related ffi libs/headers are missing...

[lelit]

Maybe I should retry to build this on top of #182?

[njsmith]

Oh, if you're getting an immediate segfault locally, that's a known issue: Debian is using a kernel with vsyscall=none. #158 is one solution, or possibly simpler is to edit your kernel command line to add vsyscall=emulate. (At least I think that's what it is; googling vsyscall docker should find more details.)

[lelit]

Thanks for the enlightenment!

[njsmith]

Okay, yeah, so the issue is that starting in 3.7, CPython requires that a system copy of libffi be installed:

A full copy of libffi is no longer bundled for use when building the _ctypes module on non-OSX UNIX platforms. An installed copy of libffi is now required when building _ctypes on such platforms. (Contributed by Zachary Ware in bpo-27979.)

So the first thing to try is adding libffi-devel to PYTHON_COMPILE_DEPS and see if that works any better.

[lelit]

Yay, it seems working now!

[lelit]

Should I upgrade this to b5, or for some reason is this a dead path?

[mehrdada]

I am wondering what the ETA on merging this change is? We want to use the docker image to build Python3.7 gRPC packages.

[hugovk]

It would be great to have this merged (with the latest rc1, and updated later with final) so downstream users can begin prep for the Python 3.7 release.

For example, Pillow is waiting for multibuild to add 3.7, which is waiting for manylinux to add it.

Thank you!

[lelit]

I switched the setting to import 3.7.0rc1.

[njsmith]

Sorry for being slow, I've just been juggling a lot of things lately. This looks great.

[lelit]

Thank you Nathaniel!