Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Building for arm64 #1846

Closed
afinch7 opened this issue Feb 27, 2019 · 105 comments · Fixed by #22298
Closed

Building for arm64 #1846

afinch7 opened this issue Feb 27, 2019 · 105 comments · Fixed by #22298
Labels
build build system or continuous integration related feat new feature (which has been agreed to/accepted)

Comments

@afinch7
Copy link
Contributor

afinch7 commented Feb 27, 2019

Builds for arm64 are now in CI, but are not ready for release. We still need to make some changes to make the tests work with this build.
I managed to get deno to build for arm64 using the aarch64-unknown-linux-gnu target. Currently It doesn't execute successfully on a raspberry pi 3b+ running a preview image of debian buster arm64/aarch64. It gives me the following print out:

root@rpi3:~/deno-build# ./deno https://deno.land/welcome.ts
#
# Fatal error in , line 0
# unreachable code
#
#
#
#FailureMessage Object: 0xfffff58d4d10
==== C stack trace ===============================

    ./deno(+0x8ad5fc) [0xaaaaafc445fc]
    ./deno(+0x9dd72c) [0xaaaaafd7472c]
    ./deno(+0x8a9f10) [0xaaaaafc40f10]
    ./deno(+0x83aa08) [0xaaaaafbd1a08]
    ./deno(+0x69c070) [0xaaaaafa33070]
    ./deno(+0x7bdbb4) [0xaaaaafb54bb4]
    ./deno(+0x7bdb00) [0xaaaaafb54b00]
    ./deno(+0x7bc6bc) [0xaaaaafb536bc]
    ./deno(+0x7bd9f8) [0xaaaaafb549f8]
    ./deno(+0x7bbbb8) [0xaaaaafb52bb8]
    ./deno(+0x2165dc) [0xaaaaaf5ad5dc]
    ./deno(+0x7c0be4) [0xaaaaafb57be4]
    ./deno(+0x663f68) [0xaaaaaf9faf68]
    ./deno(+0x7c0124) [0xaaaaafb57124]
    ./deno(+0x1c8e80) [0xaaaaaf55fe80]
    ./deno(+0x1c8f68) [0xaaaaaf55ff68]
    ./deno(+0x1b3dec) [0xaaaaaf54adec]
    ./deno(+0xce2f0) [0xaaaaaf4652f0]
    ./deno(+0x112b78) [0xaaaaaf4a9b78]
    ./deno(+0x11ef74) [0xaaaaaf4b5f74]
    ./deno(+0xb80c38) [0xaaaaaff17c38]
    ./deno(+0xb8fcf4) [0xaaaaaff26cf4]
    ./deno(+0xb814ec) [0xaaaaaff184ec]
    ./deno(+0x112d50) [0xaaaaaf4a9d50]
    /lib/aarch64-linux-gnu/libc.so.6(__libc_start_main+0xe4) [0xffff8644ed24]
    ./deno(+0xbae4c) [0xaaaaaf451e4c]

I opened up the executable with a arm compatible disassembler, and located the last call from within deno before failure, 0x1b3dec. It looks like this call occurs here

v8::Isolate* isolate = v8::Isolate::New(params);
This leads me to suspect an issue with loading the snapshot.

I managed to get this working successfully with the changes in [my forked branch](https://github.com/afinch7/deno/tree/arm-builds).

To replicate my arm64 build you will need docker installed.
Then it should be as simple as:

  1. Add the following options to .gn
target_cpu = "arm64"
v8_target_cpu = "arm64"
  1. Add the following to Dockerfile in your deno repo root
FROM ubuntu:bionic

RUN apt update && apt install -y curl

RUN curl https://sh.rustup.rs -sSf | sh -s -- -y

RUN /root/.cargo/bin/rustup target add aarch64-unknown-linux-gnu

RUN echo "[target.aarch64-unknown-linux-gnu]\nlinker = \"aarch64-linux-gnu-gcc\"" > /root/.cargo/config

RUN apt install -y nodejs yarn python build-essential cmake make

RUN apt -yq --no-install-suggests --no-install-recommends install g++-aarch64-linux-gnu gcc-aarch64-linux-gnu g++-7-multilib libc6-arm64-cross

WORKDIR /deno

RUN echo "source $HOME/.cargo/env" > /root/.bash

CMD ["/bin/bash", "--init-file", "/root/.bash"]
  1. Run docker build -f Dockerfile -t deno-arm64-build . && docker run -it -v "$PWD/:/deno" -u $(id -u root):$(id -g ${USER}) deno-arm64-build in your deno repo root. This should build a docker image with the required tools and drop you into a bash shell on said image with your deno repo mounted at /deno.
  2. From this point you simply need to run cargo build --target=aarch64-unknown-linux-gnu --release
  3. Once the build completes the executable should be located at target/aarch64-unknown-linux-gnu/release/deno

I wasn't able to get a build working with tools/build.py, since I couldn't find a way to specify a target for rust it might be possible to use env vars like CC, AR, CFLAGS, etc. I might try that at some point, but I'm putting my efforts into figuring out if the snapshots are even being included in the build correctly right now.
I not sure what changed, but I can't get the build to work anymore with docker. I can still get this to work outside of docker though. I'm working on getting this into CI see #1887. The main problem with getting this into CI is how long it takes to build.

Edit: fixed some issues with the Dockerfile
Edit: added the current user group to the docker run command so this doesn't break your repo as much
Edit: my forked branch has modifications to make this work now

@afinch7
Copy link
Contributor Author

afinch7 commented Feb 27, 2019

@ry My guess at this point is that v8 snapshots are not independent of the binaries used to generate them. That would make since considering that the snapshots are generated by the compiler host using a native binary. Can you confirm this is the case with v8?

@ry
Copy link
Member

ry commented Feb 27, 2019

@afinch7 I'll have to dig into this later when I have some time - but one random guess - try v8_multi_arch_build = true

@afinch7
Copy link
Contributor Author

afinch7 commented Feb 27, 2019

@ry Turns out I was right chromium has had a similar issue with cross compiled builds using snapshots. My plan at this point remove inclusion of compile time generated snapshots on cross compiled builds, and setup some logic to allow them to generate the snapshots during the first runtime(and store them on disk for next time). I would build on aarch64, but depot_tools doesn't seem to have a gn binary for aarch64. #1852

@ry
Copy link
Member

ry commented Mar 19, 2019

basic arm support added in 8c7a12d

@Trucoto
Copy link

Trucoto commented Jun 21, 2020

I have a project where deno would fit perfectly, I made some tests on x86_64 and it works well, but I will need arm support in the future, so does this have a place in the roadmap? What is the current status? What does it mean "basic support"?

@kitsonk
Copy link
Contributor

kitsonk commented Nov 3, 2020

ARM64 support was reverted some time ago... re-opening.

@kitsonk kitsonk reopened this Nov 3, 2020
@kitsonk kitsonk added feat new feature (which has been agreed to/accepted) build build system or continuous integration related labels Nov 3, 2020
@leolivier
Copy link

FYI, not sure if deno is using Travis CI in conjunction with github but Travis CI now supports multi arch including arm64, see https://docs.travis-ci.com/user/multi-cpu-architectures/ and you don't need to use qemu for that (as far as I understood, I didn't test it by myself)

@Dygear
Copy link
Contributor

Dygear commented Nov 11, 2020

I've also had zero issues building an aarch64 build for the last couple of versions that I've tested. I'd be happy to donate a few Raspberry Pi 4 (8GB) to this project in order to make a stable release happen. Now that the Mac has released their ARM hardware, I can also build the same on there for that.

@kitsonk
Copy link
Contributor

kitsonk commented Nov 11, 2020

FYI, not sure if deno is using Travis CI in conjunction with github but Travis CI

We use GitHub Actions exclusively. We are working with GitHub on the potential of their ARM64 support.

I'd be happy to donate a few Raspberry Pi 4 (8GB) to this project in order to make a stable release happen.

That isn't necessary, and likely wouldn't solve the problem of integrating it reliably into a repeatable and supportable APIs.

@frankhale
Copy link

For those that may not know Deno builds easily on a Pi 4 without any modifications necessary. Takes about an hour on a non-overclocked Pi 4 with just a microSD card. Have fun!

@jimmont
Copy link

jimmont commented Jan 17, 2021

Echoing what frankhale mentioned, the Raspberry Pi OS 64bit-beta updated on the RPi4 8GB installs in about 1 hour with standard install and cargo install deno --locked, from there deno works as expected. (RPi3b gets stuck somewhere in my 2 attempts so-far, letting it run overnight, etc.) Haven't tried 32bit Raspberry Pi OS on this RPI4 device (yet).

@LukeChannings
Copy link
Contributor

I have had some success building for ARM64 on GitHub Actions. It takes about 2 hours, compared with 20 minutes for AMD64.

https://github.com/LukeChannings/docker-deno/actions/runs/482795009

The process for this repo is:

That last step is failing and I haven't had the time to finish it, but everything else works, and there's a published 1.6.3 image under lukechannings/deno:1.6.3 if anyone needs it.

I built this for another project I'm working on, and people seem to be running the ARM64 image on a Raspberry Pi 4 without problems.
LukeChannings/moviematch#27

@frankhale
Copy link

@jimmont I made it almost the entire way through the building process on the RPi3B+ but I came to a point where it would lock up consistently. I'm guessing it's running out of memory? I was building it on Ubuntu Server 20.10 so the Pi would not be using much memory. I also made a 4 gb swap file which didn't seem to help.

@Dygear
Copy link
Contributor

Dygear commented Jan 18, 2021

I'm willing to bet that you are running out of memory. Even on a Raspberry Pi 4 4GB, you need to use swap space (around 2GB) in order to compile correctly. I think one compile thread uses more than the 4GB boundary of a single application thread in the 32bit OS. So we are gonna have a hard time here. I've been using an 8GB Pi 4 and that just squeaks by. I'll try later tonight if an 8GB Pi can compile on the 32bit OS or and confirm if it's a memory boundary problem.

@frankhale
Copy link

Has anyone attempted a compile of 1.7.0 on a Pi 4? I did but I'm running into problems:

error: failed to add native library /home/ubuntu/code/deno/target/release/gn_out/obj/librusty_v8.a: file too small to be an archive

error: aborting due to previous error

error: could not compile `rusty_v8`

@LukeChannings
Copy link
Contributor

There are no rusty_v8 aarch64 builds yet. See denoland/rusty_v8#589

@lucacasonato
Copy link
Member

You can try to build fully from scratch by setting the V8_FROM_SOURCE env var to 1. Note that this will take a very long time.

@secext2022
Copy link
Contributor

Another method:

If we just run build.rs through QEMU aarch64 (just to generate the snapshot), we can do cross compile on x86 host, and it is fast (about 34 minutes on github action).

#19759

@runlevel5
Copy link

Cross-compilation can be done via QEMU aarch64 emulation. It is not as performant as the Apple Silicon but hey it should be around 30min

@z-nerd
Copy link

z-nerd commented Aug 20, 2023

Hi I need to use deno in termux! "Deno for ARM64" project work fine in proot-distro but not in native termux and compile didn't work in either of them. Please add support to aarch64 in compile and in install.

@secext2022
Copy link
Contributor

Hi I need to use deno in termux! "Deno for ARM64" project work fine in proot-distro but not in native termux and compile didn't work in either of them. Please add support to aarch64 in compile and in install.

@zero-js-dev

You can try this: https://github.com/fm-elpac/v8-src/tree/deno-1.36

deno 1.36.2 for aarch64-linux-android and x86_64-linux-android. It should work in termux.

@secext2022
Copy link
Contributor

Another method:

If we just run build.rs through QEMU aarch64 (just to generate the snapshot), we can do cross compile on x86 host, and it is fast (about 34 minutes on github action).

#19759

With a modified cargo, we can cross build deno for aarch64-unknown-linux-gnu in github actions CI (host: x86_64-unknown-linux-gnu).

The modified cargo will only run build.rs (build-script-build) through QEMU docker (aarch64), all other is build on host (x86_64), so it is fast (about 35 minutes build time).

We only need a little patch:

diff '--color=auto' -ru -x Cargo.lock workdir-1/deno-1.36.2/build.rs workdir/deno-1.36.2/build.rs
--- workdir-1/deno-1.36.2/build.rs	2023-08-24 19:38:40.831557143 +0800
+++ workdir/deno-1.36.2/build.rs	2023-08-24 19:39:43.532680484 +0800
@@ -400,9 +400,6 @@
   // Host snapshots won't work when cross compiling.
   let target = env::var("TARGET").unwrap();
   let host = env::var("HOST").unwrap();
-  if target != host {
-    panic!("Cross compiling with snapshot is not supported.");
-  }
 
   let symbols_path = std::path::Path::new("napi").join(
     format!("generated_symbol_exports_list_{}.def", env::consts::OS).as_str(),

all other is just work.

https://github.com/fm-elpac/v8-src/tree/deno-1.36

@z-nerd
Copy link

z-nerd commented Aug 29, 2023

@secext2022 Thanks a lot ❤️ this build of yours "aarch64-linux-android" works(run and compile) on my termux without to need use additional proot-distro.

If you had time please make it installable and upgradable like below project(for everyone to easy using it)

curl -s https://gist.githubusercontent.com/LukeChannings/09d53f5c364391042186518c8598b85e/raw/ac8cd8c675b985edd4b3e16df63ffef14d1f0e24/deno_install.sh | sh

@runlevel5
Copy link

If the core team is looking for a powerful machine for development, please have a look at

https://www.adlinktech.com/en/News/com-hpc-ampere-altra-developer-platform-arm-sr

@CyberFlameGO
Copy link

When the feature in this changelog entry goes into GA I think this issue is ready to be addressed with a PR :) Hope this helps!!

@mgenereu
Copy link

mgenereu commented Nov 2, 2023

https://github.blog/changelog/2023-10-30-accelerate-your-ci-cd-with-arm-based-hosted-runners-in-github-actions/

@approached
Copy link

Do you have tried this:

https://github.com/LukeChannings/deno-arm64/releases

@06kellyjac
Copy link
Contributor

Friendly reminder (as it's been about 2 years since I last brought it up) nixpkgs has been building deno for:

Pre-built releases are available via the cache or you can build again from source if you'd like too.
Due to sandboxing and dependency resolution if it is building on hydra (see links above) it's essentially guaranteed it will build for you (if you use the same commit of nixpkgs)

@bartlomieju
Copy link
Member

FYI we've applied for Linux ARM runners at GitHub. As soon as we get access to them we will work on providing first-class Linux ARM builds for releases.

@approached
Copy link

Also available on docker hub:

https://hub.docker.com/r/lukechannings/deno

@bartlomieju
Copy link
Member

We are now actively working towards adding support for the official Linux ARM builds.

No promises on the dates, but stay tuned 🤞

@bartlomieju
Copy link
Member

👀 #22298

mmastrac added a commit that referenced this issue Feb 7, 2024
This implements officially blessed and tested deno binaries for ARM64. 

Thanks to @LukeChannings for his tireless work in maintaining the
deno-arm64 [1] repo, without which this project would have been far more
complicated. For those of you requiring support for older GLIBC
versions, that repo may still be required for the near future.

Limitations: 

- This initial build is built on Ubuntu 22 using the stock GLIBC, which
will limit the utility of these binaries in certain use-cases (eg: early
versions of Ubuntu). We will attempt to support earlier versions of
ARM64 GLIBC in a later revision.
- Like the stock Linux x64 build, this is not a static build and
requires GLIBC. Running on Alpine will require installation of GLIBC.
 
Fixes #1846, #4862 

[1] https://github.com/LukeChannings/deno-arm64
littledivy pushed a commit that referenced this issue Feb 8, 2024
This implements officially blessed and tested deno binaries for ARM64. 

Thanks to @LukeChannings for his tireless work in maintaining the
deno-arm64 [1] repo, without which this project would have been far more
complicated. For those of you requiring support for older GLIBC
versions, that repo may still be required for the near future.

Limitations: 

- This initial build is built on Ubuntu 22 using the stock GLIBC, which
will limit the utility of these binaries in certain use-cases (eg: early
versions of Ubuntu). We will attempt to support earlier versions of
ARM64 GLIBC in a later revision.
- Like the stock Linux x64 build, this is not a static build and
requires GLIBC. Running on Alpine will require installation of GLIBC.
 
Fixes #1846, #4862 

[1] https://github.com/LukeChannings/deno-arm64
@kuboon
Copy link

kuboon commented Feb 9, 2024

I ran curl -fsSL https://deno.land/install.sh | sh and got message

Error: Official Deno builds for Linux aarch64 are not available. (see: https://github.com/denoland/deno/issues/1846 )

https://deno.land/install.sh should be updated.

@littledivy
Copy link
Member

Updated. curl -fsSL https://deno.land/install.sh | sh should work now

@z-nerd
Copy link

z-nerd commented Feb 9, 2024

Updated. curl -fsSL https://deno.land/install.sh | sh should work now

Deno didn't care about its community at all. 1 year ago I requested this simple build but still didn't work(install but don't work). Thanks to @secext2022 who responded me immediately that time and give me a build that work

@z-nerd
Copy link

z-nerd commented Feb 9, 2024

Hi I need to use deno in termux! "Deno for ARM64" project work fine in proot-distro but not in native termux and compile didn't work in either of them. Please add support to aarch64 in compile and in install.

@zero-js-dev

You can try this: https://github.com/fm-elpac/v8-src/tree/deno-1.36

deno 1.36.2 for aarch64-linux-android and x86_64-linux-android. It should work in termux.

This build is old but works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build build system or continuous integration related feat new feature (which has been agreed to/accepted)
Projects
None yet
Development

Successfully merging a pull request may close this issue.