Document dependency on libssl when compiling Julia from source

[jiahao]

On a Ubuntu 14.04 machine with Julia master and no package repo, Pkg.init gives an error:

julia> Pkg.init()
INFO: Initializing package repository /afs/csail.mit.edu/u/j/jiahao/.julia/v0.5
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl
ERROR: [Code:ERROR, Class:Net]: Unsupported URL protocol
 [inlined code] from libgit2/error.jl:96
 in clone at libgit2/repository.jl:95
 in clone at libgit2.jl:303
 in anonymous at pkg/dir.jl:57
 in cd at file.jl:22
 in init at pkg/dir.jl:55
 in init at pkg.jl:23

[yuyichao]

Is this the issue when libssl is not installed?

[wildart]

Yep, no libssl.

[tkelman]

apt-get install libssl-dev.

openssl-devel on fedora. we should probably put these in the readme

edit: also do make -C deps distclean-libgit2 to force a reconfigure

[jiahao]

I can confirm that the new machine had no libssl installed. Problem went away after installing libssl-dev and recompiling Julia. I think we can change this to a documentation issue.

[jiahao]

Also would be nice to have an error message that explained what the problem was and how to fix it.

[tkelman]

Unsupported URL protocol, of course. If you had run the libgit2 and/or pkg tests with a non-ssl build I suspect they would have failed.

[JeffBezanson]

Do we really want to add more manually-installed dependencies?

[tkelman]

Do we want to be able to clone packages securely? Previously git was a manually-installed dependency for Pkg, needing just the openssl headers at compile time is much smaller. The alternatives are to put in build instructions for openssl into deps/Makefile (which would be a mess), or work on a PR to libgit2 to add support for mbedtls and build that instead. Mbedtls has less license uncertainty so we may want to pursue this anyway.

[wildart]

@tkelman Maybe we can allow through some build flag linking to libssh?

[tkelman]

I don't think libssh is the issue here, https is more important to support as mandatory than ssh (typo)

[wildart]

Sorry, got confused. Why would it be a problem to list openssl as a dependency? Majority of platforms supports it.

[tkelman]

One more thing that needs to be installed before make gives you a working build is Jeff's concern I imagine. I don't think it's a problematic one as build deps go.

edit: given the important caveat that it's only needed on linux

[nalimilan]

Doesn't sound like a high barrier, at least if a clear error can be printed at build time instead of raising a relatively obscure error at run time.

[tkelman]

Would have to look into whether it's possible in libgit2's cmakelists to error if openssl is not found on linux. May need a patch to do that.

[axsk]

Is there a workaround when libssl is not installed and I don't have root access to the machine to install it / since which commit is libssl necessary?

[tkelman]

libssl is necessary on linux since merging #11196, which switched the package manager to use libgit2 instead of command-line git. You might be able to get openssl and headers as non-root from conda, then add a libgit2 build flag to point at that libssl.

[maxj]

Looks like the Makefile also assumes that your cert is named cert.pem

cp $(shell openssl version -d | cut -d '"' -f 2)/cert.pem $(DESTDIR)$(datarootdir)/julia/

On a new install of Ubuntu installing libssh-dev and creating a symlink in /usr/lib/ssl to an actual .pem file resolved this issue.

[Keno]

Is there a guide on what to do on a mac? Not being able to tag packages is a little unfortunate.

[StefanKarpinski]

I don't recall you ever tagging any of your packages, @Keno :-P

[Keno]

You're right. I do however, submit pull requests which uses the same mechanism.

[tkelman]

Where does libssh usually come from on mac? Does apple install the dev headers for it by default or with the xcode command line tools? If not, homebrew?

[axsk]

I am now on the second machine without root-access and non available libssl-dev, and thus cannot run Julia in my work environment so far.

I think the dependency on libssl-dev should be optional.

[tkelman]

See #10763 (comment). 0.5-dev should not be used by anyone except for active development, aka submitting pull requests, at this time. If you really want to use bleeding edge Julia for whatever reason and don't have root access to install the build dependencies, you can use nightly binaries.

[Keno]

FWIW, this still doesn't work for me, even if I install libssh2 using Homebrew.

[tkelman]

This issue is about libssl, not libssh. Use https remotes going forward.

[Keno]

But I like ssh remotes. Saves me from entering my password everywhere.

[tkelman]

You have to enter a passphrase to unlock an ssh key. And remember to start an agent in every terminal you start Julia from. I fail to see how ssh is better, and it requires building and shipping yet another library.

[Keno]

I'm not asking it to be shipped. I would just like it to work if I have libssh installed. Also, on OS X the ssh-agent is handled through the key chain.

[tkelman]

I believe you need to comment out the line that overrides PKG_CONFIG_LIBDIR in Make.inc.