Add official Dockerfile #17

pm47 · 2024-03-26T20:25:42Z

Follow-up to #1.

I think building native binaries on docker is not the right approach. It's better to run on the jvm: easier build, faster runtime and compatible with linux arm64, which we don't support otherwise.

TODO:

remove lightning-kmp build once Fix mempool.space ios tests lightning-kmp#622 is merged
remove RUN mkdir .phoenix once Factor initialization of datadir directory #15 is merged

cc @sethforprivacy

sethforprivacy · 2024-03-26T21:00:06Z

Will take a deeper look asap, but at first glance I love this approach!

tsmith123 · 2024-03-26T22:01:03Z

I'm getting "ERROR: failed to solve: eclipse-temurin:17-jre-alpine: no match for platform in manifest: not found" running on an M2 Macbook when building image.

pm47 · 2024-03-26T22:04:18Z

Try with eclipse-temurin:21-jre-alpine Le mar. 26 mars 2024, 23:01, Tom Smith ***@***.***> a écrit :

…

I'm getting "ERROR: failed to solve: eclipse-temurin:17-jre-alpine: no match for platform in manifest: not found" running on an M2 Macbook — Reply to this email directly, view it on GitHub <#17 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAPPFPRMQKUBXHNBG3FGC4LY2HO2LAVCNFSM6AAAAABFJUNRIOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRRGU2DKOBQGE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

tsmith123 · 2024-03-26T22:12:19Z

Changed both to 21 and it seems better but now I'm getting:

"ERROR: failed to solve: process "/bin/sh -c git clone --recursive --branch ${LIGHTNING_KMP_BRANCH} https://github.com/ACINQ/lightning-kmp . && test git rev-parse HEAD = ${LIGHTNING_KMP_COMMIT_HASH} || exit 1 && ./gradlew publishToMavenLocal -x dokkaHtml" did not complete successfully: exit code: 1"

pm47 · 2024-03-27T13:50:29Z

Rebased on master.

Changed both to 21 and it seems better but now I'm getting:
"ERROR: failed to solve: process "/bin/sh -c git clone --recursive --branch ${LIGHTNING_KMP_BRANCH} https://github.com/ACINQ/lightning-kmp . && test git rev-parse HEAD = ${LIGHTNING_KMP_COMMIT_HASH} || exit 1 && ./gradlew publishToMavenLocal -x dokkaHtml" did not complete successfully: exit code: 1"

@tsmith123 your issue should be fixed at 5630fa0.

sstone

Building this docker image still won't work on macOS arm64 because of gradle/gradle#24875.
It could be fixed by using a glibc-based distro (ubuntu, ...) or maybe switching to a snapshot version of gradle 8.8 but I'm not sure it's worth it as macOS arm64 is not a production target (and we already publish a native phoenixd exe for macOS arm64).

pm47 · 2024-03-27T17:44:21Z

So @sstone should work on all 4 combinations linux/macos x64/arm64?

More portability is better for docker I think. Besides, ubuntu 21 + jdk21 seems tailor made for a bitcoin app 😄

pm47 · 2024-03-27T17:54:25Z

Actually went back to alpine for the final image. Best of both worlds or over the top?

sethforprivacy · 2024-03-27T19:13:21Z

Dockerfile looks GTG to me, fantasticly simplified approach compared to building from source.

I'll switch to using this as well in mine (will keep my repo just so there can be a published image in GHCR/Dockerhub) with proper credit, of course. Thanks for all of the work on this @pm47!

sethforprivacy · 2024-03-27T19:22:59Z

I have a few other minor improvements (none necessary, just tweaks and more explicit declaration of things like user:group pairings) in my PR on my own repo here, feel free to take anything that strikes your fancy, no credit etc. necessary.

sethforprivacy/phoenixd-docker#3

sethforprivacy · 2024-03-27T19:32:19Z

Sorry for the message spam, but I actually just caught a breaking issue with the proposed Dockerfile. This Dockerfile doesn't properly set the file permissions for /phoenix which can sometimes cause the image to error out with a permissions issue when run.

See the fix here: sethforprivacy/phoenixd-docker@f3a63e0

sethforprivacy · 2024-03-27T19:47:54Z

~~@pm47 seems there is a build issue for ARMv8, can't get it to find a workable Kotlin version. Logs:~~

~~https://paste.sethforprivacy.com/?ddbbbdc28345c06d#4DPdDZvXT4CGRBJ7jo1CDnNTUB3Ynkvp5yuMRcMJ3iqX~~

~~Not sure if you can access it, but the full Github Action run is here: https://github.com/sethforprivacy/phoenixd-docker/actions/runs/8457575444/job/23169856878?pr=3~~

Edit: Nevermind, seems you can't build ARMv8 using ARMv8 due to Kotlin having no native aarch64 Linux support. Will find another way to distribute ARMv8 images then.

pm47 · 2024-03-27T20:40:57Z

Edit: Nevermind, seems you can't build ARMv8 using ARMv8 due to Kotlin having no native aarch64 Linux support. Will find another way to distribute ARMv8 images then.

On the contrary, it should work, that is the point of using a jvm build for docker instead of a native one. Seems like gradle tries to build the native version, which we are not going to use anyway.

Will look at your other changes, feel free to suggest changes directly on this PR too!

sethforprivacy

I think this encompasses all of my minor changes/improvements, feel free to take or leave any and all of them as you see fit :)

.docker/Dockerfile

sethforprivacy · 2024-03-27T21:01:30Z

.docker/Dockerfile

+RUN apt-get update \
+    && apt-get upgrade -y
+RUN apt-get install -y --no-install-recommends bash git \
+    && apt clean


Better apt cleanup:

Suggested change

&& apt clean

&& apt clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

.docker/Dockerfile

Co-authored-by: Seth For Privacy <40500387+sethforprivacy@users.noreply.github.com>

sstone · 2024-03-28T12:34:18Z

Building phoenixd in docker is hard and there does not seem to be a way to make it work for linux/arm64 (which would be a real production target), even if issues with gradle on alpine are solved.
Why not remove the build step and have the docker file simply download the released JVM binaries and use them directly ? that would give us a simple docker file that would work on all targets the JVM version supports (including macos arm64/x64 and linux arm64/x64) ?

sstone · 2024-03-28T13:26:23Z

Building phoenixd in docker is hard and there does not seem to be a way to make it work for linux/arm64

@pm47 found a workaround (see #25)

pm47 · 2024-03-28T13:26:45Z

@sethforprivacy the build should pass on arm at 75247e4 (current master tip)

sethforprivacy · 2024-03-28T13:37:57Z

@sethforprivacy the build should pass on arm at 75247e4 (current master tip)

Testing now! https://github.com/sethforprivacy/phoenixd-docker/actions/runs/8468177452/job/23200542411?pr=3

sethforprivacy · 2024-03-28T14:41:11Z

@sethforprivacy the build should pass on arm at 75247e4 (current master tip)

Builds are working on ARMv8 now!

.docker/Dockerfile

Co-authored-by: Seth For Privacy <40500387+sethforprivacy@users.noreply.github.com>

pm47 · 2024-03-28T14:59:51Z

Looking pretty good I think. Will update branch/version after next release

tsmith123 · 2024-03-29T08:36:40Z

Not sure if this Dockerfile is in its final state yet but it doesn't work for me on an M2 Macbook.

sethforprivacy · 2024-03-29T12:18:48Z

Not sure if this Dockerfile is in its final state yet but it doesn't work for me on an M2 Macbook.

This issue was fixed in master, so next release will include the fix (per #17 (comment)). It was discussed at length above if you want more details on it!

pm47 · 2024-03-29T13:59:27Z

@tsmith123 Please confirm you are getting this error when using the Dockerfile at commit f255a11.

dnjooiopa · 2024-03-30T04:33:47Z

@tsmith123 Please confirm you are getting this error when using the Dockerfile at commit f255a11.

I got the same error on an M1 MacBook.
Commit f255a11

pm47 · 2024-04-04T21:31:41Z

@tsmith123 @dnjooiopa Please try again at 1c3a7ab thanks!

dnjooiopa · 2024-04-06T02:06:56Z

@tsmith123 @dnjooiopa Please try again at 1c3a7ab thanks!

Great! Docker is now successfully built on my M1 Mac.

sethforprivacy · 2024-04-10T15:53:05Z

Amazing work on this, thank you @pm47! I've merged a very similar Dockerfile with automatic builds via Github Actions, so anyone who wants to use a ready-built image for phoenixd can use it easily:

https://github.com/sethforprivacy/phoenixd-docker/pkgs/container/phoenixd

The latest v0.1.4 version is building now and will be available shortly.

The Dockerfile builds on windows x86, linux x86/arm and mac x86/arm. It uses a JVM build, which is simpler and more portable than native builds. --------- Co-authored-by: Seth For Privacy <40500387+sethforprivacy@users.noreply.github.com>

pm47 requested a review from sstone March 26, 2024 20:25

pm47 added 3 commits March 27, 2024 14:42

add dockerfile

053a97c

remove annoying git warning

bf1eb0b

remove lightning-kmp build and use jdk21

5630fa0

pm47 force-pushed the docker branch from ad66df4 to 5630fa0 Compare March 27, 2024 13:49

pm47 marked this pull request as ready for review March 27, 2024 13:50

pm47 added 2 commits March 27, 2024 14:58

more simplifications

52febf5

move Dockerfile to .docker

2eb26c3

sstone reviewed Mar 27, 2024

View reviewed changes

use ubuntu instead of alpine

64d2226

pm47 added 2 commits March 27, 2024 18:49

fixup! use ubuntu instead of alpine

2a96880

keep alpine image for the final image

ce515f7

sethforprivacy mentioned this pull request Mar 27, 2024

Migrate to upstream Dockerfile courtesy of @pm47 sethforprivacy/phoenixd-docker#3

Merged

sethforprivacy suggested changes Mar 27, 2024

View reviewed changes

apply suggestions from @sethforprivacy

8bb63ae

Co-authored-by: Seth For Privacy <40500387+sethforprivacy@users.noreply.github.com>

sstone mentioned this pull request Mar 28, 2024

Do not add linux target on aarch64 architectures #25

Merged

sethforprivacy reviewed Mar 28, 2024

View reviewed changes

.docker/Dockerfile Outdated Show resolved Hide resolved

mount /phoenix, not /phoenix/.phoenix

f255a11

Co-authored-by: Seth For Privacy <40500387+sethforprivacy@users.noreply.github.com>

use branch aarch64-disable-native

1c3a7ab

use v0.1.4 release

feb89da

pm47 merged commit 2964e34 into master Apr 10, 2024

pm47 deleted the docker branch April 10, 2024 15:43

pm47 mentioned this pull request May 13, 2024

Add docker build for linux release. #46

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add official Dockerfile #17

Add official Dockerfile #17

pm47 commented Mar 26, 2024 •

edited

Loading

sethforprivacy commented Mar 26, 2024

tsmith123 commented Mar 26, 2024 •

edited

Loading

pm47 commented Mar 26, 2024 via email

tsmith123 commented Mar 26, 2024

pm47 commented Mar 27, 2024

sstone left a comment •

edited

Loading

pm47 commented Mar 27, 2024

pm47 commented Mar 27, 2024

sethforprivacy commented Mar 27, 2024

sethforprivacy commented Mar 27, 2024

sethforprivacy commented Mar 27, 2024

sethforprivacy commented Mar 27, 2024 •

edited

Loading

pm47 commented Mar 27, 2024

sethforprivacy left a comment

sethforprivacy Mar 27, 2024

sstone commented Mar 28, 2024

sstone commented Mar 28, 2024

pm47 commented Mar 28, 2024

sethforprivacy commented Mar 28, 2024

sethforprivacy commented Mar 28, 2024

pm47 commented Mar 28, 2024

tsmith123 commented Mar 29, 2024

sethforprivacy commented Mar 29, 2024

pm47 commented Mar 29, 2024

dnjooiopa commented Mar 30, 2024

pm47 commented Apr 4, 2024 •

edited

Loading

dnjooiopa commented Apr 6, 2024 •

edited

Loading

sethforprivacy commented Apr 10, 2024

	&& apt clean
	&& apt clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

Add official Dockerfile #17

Add official Dockerfile #17

Conversation

pm47 commented Mar 26, 2024 • edited Loading

sethforprivacy commented Mar 26, 2024

tsmith123 commented Mar 26, 2024 • edited Loading

pm47 commented Mar 26, 2024 via email

tsmith123 commented Mar 26, 2024

pm47 commented Mar 27, 2024

sstone left a comment • edited Loading

Choose a reason for hiding this comment

pm47 commented Mar 27, 2024

pm47 commented Mar 27, 2024

sethforprivacy commented Mar 27, 2024

sethforprivacy commented Mar 27, 2024

sethforprivacy commented Mar 27, 2024

sethforprivacy commented Mar 27, 2024 • edited Loading

pm47 commented Mar 27, 2024

sethforprivacy left a comment

Choose a reason for hiding this comment

sethforprivacy Mar 27, 2024

Choose a reason for hiding this comment

sstone commented Mar 28, 2024

sstone commented Mar 28, 2024

pm47 commented Mar 28, 2024

sethforprivacy commented Mar 28, 2024

sethforprivacy commented Mar 28, 2024

pm47 commented Mar 28, 2024

tsmith123 commented Mar 29, 2024

sethforprivacy commented Mar 29, 2024

pm47 commented Mar 29, 2024

dnjooiopa commented Mar 30, 2024

pm47 commented Apr 4, 2024 • edited Loading

dnjooiopa commented Apr 6, 2024 • edited Loading

sethforprivacy commented Apr 10, 2024

pm47 commented Mar 26, 2024 •

edited

Loading

tsmith123 commented Mar 26, 2024 •

edited

Loading

sstone left a comment •

edited

Loading

sethforprivacy commented Mar 27, 2024 •

edited

Loading

pm47 commented Apr 4, 2024 •

edited

Loading

dnjooiopa commented Apr 6, 2024 •

edited

Loading