[release-0.4, RFC] backports for 0.4.2

[tkelman]

cc @ViralBShah still working on this, putting up my progress so far more or less ready I think, running PkgEval now

[amitmurthy]

I just restarted the Travis build to see if it shows up again.

[tkelman]

Something backported on this branch introduces an intermittent failure, since eccf231 succeeded make testall 50 times in a row locally, but 74dcb33 had several failures out of the same number of runs. Needs a bisect and a revert.

[Keno]

Which OS?

[tkelman]

Linux 64. It's a bit unreliable to bisect since it happens somewhat rarely, but my suspicion is it's most likely 8b3c67b

[JeffBezanson]

Let's also make sure this doesn't exhibit #14245.

[amitmurthy]

Is it the same failure every time?

[tkelman]

Not sure. These logs are long and slightly out of order because of the redirection to get stdout and stderr to go to the same file: https://gist.github.com/edeec37dc129f2a5c2f2

There are some conversion errors and bounds errors here https://gist.githubusercontent.com/anonymous/edeec37dc129f2a5c2f2/raw/51db4fbb6d7895b723fe5b6994d19539fe8d0f7c/tests-74dcb330dd76ecad44739dac36229616d71ce557.log

and an UndefRefError here https://gist.githubusercontent.com/anonymous/edeec37dc129f2a5c2f2/raw/51db4fbb6d7895b723fe5b6994d19539fe8d0f7c/tests-cbebbb71b58531c6aa00906188363886955acaa0.log

[amitmurthy]

8b3c67b is a hygiene fix. We can try reverting that, but the issue actually seems to be a memory corruption.

[amitmurthy]

FWIW, for i in seq 1 50; do ../julia runtests.jl parallel; done just went through on a local Linux 64 machine. Since the parallel test spawns out parallel_exec.jl, the original UndefRef error should have shown up.

Something in the Travis environment is triggering this?

[tkelman]

Or possibly something caused by a different earlier test executed on the same worker?

[amitmurthy]

That does not hold true for the parallel test.

julia/test/parallel.jl

Lines 14 to 16 in 071390a

	cmd = `$(Base.julia_cmd()) $inline_flag $cov_flag --check-bounds=yes --depwarn=error parallel_exec.jl`
	if !success(pipeline(cmd; stdout=STDOUT, stderr=STDERR)) && ccall(:jl_running_on_valgrind,Cint,()) == 0

parallel_exec.jl is run outside the initial set of julia processes.

[amitmurthy]

It took a few hours, but 50 runs of runtests.jl all just finished with no errors locally.

  | | |_| | | | (_| |  |  Version 0.4.2-pre+117 (2015-12-03 21:37 UTC)
 _/ |\__'_|_|_|\__'_|  |  tk/backports-0.4.2/74dcb33* (fork: 99 commits, 9 days)
|__/                   |  x86_64-linux-gnu

julia> versioninfo()
Julia Version 0.4.2-pre+117
Commit 74dcb33* (2015-12-03 21:37 UTC)
Platform Info:
  System: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Core(TM) i7-4770HQ CPU @ 2.20GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.3

[tkelman]

I think it might be that my openblas patch should be windows only. Needs looking into upstream to see if the next release is going to have this same issue.

[tkelman]

though until I backport #13569, travis should be using the ppa for openblas so that doesn't make much sense

[tkelman]

If this passes on Travis and AppVeyor, I think it's ready to merge and tag. If it doesn't, please copy and save the Travis logs to a gist before restarting any failed jobs.

[nalimilan]

FWIW, the Fedora/RHEL package builds fine with the current version of this PR.