Unable to use pre-built binaries on alpine linux

[ankurcha]

I am trying to use the netty-tcnative libraries for my grpc application in a docker container. When i use the prebuilt binary I get the following error:

Inconsistency detected by ld.so: dl-lookup.c: 128: check_match: Assertion `version->filename == ((void *)0) || ! _dl_name_match_p (version->filename, map)' failed!

[normanmaurer]

we release versions build on centos and debian. I guess you will need to build it by your own for alpine.

[dimitarg]

Assuming an alpine docker image which has an oracle jdk 8 installed, and netty-tcnative available under /opt/data/netty-tcnative (perhaps through a shared volume in the host, if you don't want to bother installing git on the alpine image), here is a setup which let me build netty-tcnative successfully:

MAVEN_VERSION=3.3.3 \
 && cd /usr/share \
 && wget http://archive.apache.org/dist/maven/maven-3/$MAVEN_VERSION/binaries/apache-maven-$MAVEN_VERSION-bin.tar.gz -O - | tar xzf - \
 && mv /usr/share/apache-maven-$MAVEN_VERSION /usr/share/maven \
 && ln -s /usr/share/maven/bin/mvn /usr/bin/mvn


export MAVEN_HOME=/usr/share/maven

cd /opt/data/netty-tcnative/

apk update

apk add autoconf

apk add automake

apk add libtool 

apk add alpine-sdk

apk add apr-util

apk add apr-util-dev

apk add build-base gcc abuild binutils binutils-doc gcc-doc

apk add cmake cmake-doc

apk add ninja

apk add go

, and then

mvn clean install

@ankurcha and anyone else interested, I will let you know if the end result actually works. We use netty directly though, no grpc involved in the test.

[dimitarg]

The built native jar as desribed above, under openssl-dynamic/target/netty-tcnative--linux-x86_64.jar , works for me, if I use the following, in a runtime image based FROM jeanblanchard/java:jdk-8:

run 'apk update'
run 'apk add apr-util apr-util-dev libuuid'

It's very important to change the base image only if you know what you are doing otherwise you might end up not finding libuuid during tcnative load.

Hope this helps. Hopefully I get the time to provide this in a matter more automated than github comments someday.

Cheers, Dimitar

[flozano]

This issue has Fork17 milestone assigned, does it mean it is fixed in Fork17? I can't find the proper alpine linux jar in maven for this version...

Building is a pretty complex process, and it seems alpine is gaining a lot of momentum in docker containers. It would be really great if the binaries were built by default.

[Scottmitch]

this issue has not been resolved ... as of now we don't publish artifacts built from alpine.

[dimitarg]

netty-tcnative-1.1.33.Fork15-linux-x86_64_alpine.jar.zip

Here's the result of the build procedure I described above, for fork15, if you just wanna kick the tires. This is to be used in conjunction with 4.1.0.CR5

I could try automating this via gradle and docker some time after Jun 27. Would netty be interested to host this in a netty github repo, and publish to maven with the netty coordinates?

(It's not required to have the correct coordinates for this to work, NativeLibraryLoader will find it anyways)

[flozano]

Thanks a lot for the effort @dimitarg.

For now I'll have to stick to a more standard glibc-based docker image, but I'm definitely interested in trying to run my components with alpine again when tcnative binaries are part of "official" build/publish process for netty-tcnative.

[pires]

@dimitarg thanks to your work I've automated the build of binaries for Alpine with Docker.
To whomever needs this, feel free to use and abuse.

[dimitarg]

@pires based on your work I have this automated via gradle. Thanks!
@normanmaurer If you folks decide to support this some day, we've now automated it down to a single command. It's written in gradle, though. I did not feel like writing such an involved thing in maven.

[nmittler]

Thanks @dimitarg and @pires ... This looks great!

@normanmaurer @trustin @Scottmitch perhaps we should resume the discussion of switching gradle? The simplicity of this configuration is in pretty stark contrast to the hackery required to make tcnative build in Maven. I think it's pretty clear at this point that the complexity of this project isn't well suited for Maven.

[trustin]

+1 for switching to Gradle due to its customizability and speed.

[stresler]

@pires I'm trying to compile your alpine version (thanks!) apr has moved to 1.6.2 and I can make that change, but then I stumbled across this #255

I don't have time to delve into that, but will that glibc dependency kill musl compatibility in the future?

[gyndav]

@pires do you still publish the built versions somewhere? Didn't not managed to compile it entirely tho.

[pires]

gyndav yes https://github.com/pires/netty-tcnative-alpine/releases 2.0.6 is the latest but one can easily change that.

[normanmaurer]

@pires I wonder if we maybe can "incorporate" this into netty-tcnative as docker-compose files after this has landed:

#344

While we will not publish any builds for alpine this could at least be used to build it easily by users.

WDYT ?

[anuraaga]

Personally, the overhead of maintaining a tcnative build is a net negative compared to just sticking to larger docker images. But when I look at the philosophies of Apline and Netty I'm surprised there are no official binaries - Alpine has taken off in containers just like Netty has, so official (and thus trustworthy) support seems like a really good fit. With easy building with docker, what's the reasoning for not publishing builds?

[normanmaurer]

@anuraaga just because it makes the release-overhead even bigger.... Once we have something in a docker file we can do a followup investigation if we want to publish something or not tho.

[pires]

@normanmaurer why docker-compose instead of multi-stage builds?

[normanmaurer]

I guess I am just used to it

…

Am 03.06.2018 um 11:52 schrieb Paulo Pires ***@***.***>: @normanmaurer why docker-compose instead of multi-stage builds? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[philicious]

I would like to express strong interest in an alpine-version available from maven.
Its just to cumbersome building it yourself if you do wanna use alpine images

[re-thc]

Are there any other issues besides glibc? So if we manually install glibc on Alpine it'll still work?

[ejona86]

I don't think glibc is even an issue now. I think you just need apk update && apk add libc6-compat. That's what it seemed like when I tested for grpc: https://github.com/grpc/grpc-java/blob/master/SECURITY.md#netty