Arm64 Raspberry Pi Build Failure #2176
Comments
|
I found the relevant code here: |
|
Hi, the developer, any suggestions on this issues? I am seeing the same problem when compiling filecoin-ffi. Step to produce: The errors: |
|
@dignifiedquire would you have any suggestions? or know how to contact original commit authors https://github.com/DrPeterVanNostrand or https://github.com/sean-sn to ask them how to fix it? |
|
I looked into this PR filecoin-project/sapling-crypto#10 which inserted the code in question and it appears that they added Pedersen Hash pre-comp for amd64 architecture based on the reasoning in this document https://github.com/filecoin-project/lotus/blob/master/documentation/en/sealing-procs.md and what is needed is to add it for other architectures like arm64. I found a research document that describes the implementation https://iden3-docs.readthedocs.io/en/latest/iden3_repos/research/publications/zkproof-standards-workshop-2/pedersen-hash/pedersen.html |
|
The default implementation is in 100% rust, the issue why it is not compiling properly is that there is a bug in the dection of the target architecture and when to link the assembly. |
|
I've tried to follow this guide for cross-compiling rust for raspberry pi on linux but I have been unsuccessful |
|
@deefactorial this should fix it: filecoin-project/sapling-crypto#18 |
|
Awesome, I will test asap. |
|
There is some more work needed: filecoin-project/rust-fil-proofs#1204 |
|
Related rust-fil-proofs epic issue: filecoin-project/rust-fil-proofs#1205 |
|
@deefactorial using the awesome work above, I am currently able to build the entirety of ./lotus using: Please try this and report back when time permits! |
|
@ribasushi awesome! Its almost there I think, but I'm still getting a failure :( |
|
@RobQuistNL What arch is your pi? The underlying cryptoprimitive implementations really need a 64bit register set. There may be a way to recode the internals to fallback on 32bit-based emulation, but I am pretty sure the crypto-team has zero bandwidth to attempt this. TLDR: At present Patches welcome though: there is a decent test suite, so if you try to dig into this - there is a decent amount of guard-railing! |
|
Hey, its a |
|
@RobQuistNL apologies I used too much jargon: https://en.wikipedia.org/wiki/List_of_ARM_microarchitectures |
|
@ribasushi @deefactorial Nice work! I verified the steps mentioned above, the build is successful. But there is some warnings, I don't know if I should worry about: |
|
@LyleLee aaah, check - makes sense. Is the aarch64 only available on older models then? I should have a RPI3 and an RPI2 laying around somewhere.. |
|
I think it is an issue of the operating system. There ares some OS options for 64bit raspberry pi out there asfaict from a google search |
|
@RobQuistNL yes, @dignifiedquire is correct, I missed your raspi model. The 4 definitely has a v8 cpu: https://en.wikipedia.org/wiki/Raspberry_Pi#Specifications I suspect you need to get a "64bit OS" or somesuch. @LyleLee can you share your |
|
I'm running mine on a 64bit ubuntu server instance and it seems to have compiled. I'm just getting some permission errors, I will have to install |
|
Success! root@ubuntu:~/lotus# lscpu
Architecture: aarch64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 1
Vendor ID: ARM
Model: 3
Model name: Cortex-A72
Stepping: r0p3
CPU max MHz: 1500.0000
CPU min MHz: 600.0000
BogoMIPS: 108.00
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Vulnerable
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected
Flags: fp asimd evtstrm crc32 cpuid
root@ubuntu:~/lotus# ./lotus --version
lotus version 0.4.1+git.24d8a84a.dirtySo applying the patch above on a ubuntu server 64 bit instance as root did the trick. |
|
great news @deefactorial 🎉 |
|
@RobQuistNL I am not using a Raspberry Pi actually, but an ARM64 server. |
|
Aaah that makes sense :) Its time to phase out x86 anyway and go for ARM ;p |
|
Really wondering what |
There are 512MiB benches if you scroll higher up, the 32/64GiB will show up Mon-ish, check the repo for new commits |
|
wow PC1 is so much faster :O really wonder what the 32gb looks like. To compare: Looks like ARM is going to be the way to go for PC1.... Awesome data! The sha256 is actually slower on that ARM compared to my 3900X, so it might have to do with something else. I get about 2.2M for 16kb whereas your machine shows 1.5M for 16kb. Both still much faster than some intel x86 tho |
It's Jeff Bezos' machine :) The reason I compare the boxes core-to-core within AWS is because they both sit behind a common set of "handicaps" ( various vuln. mitigations, hypervisors, etc ). There is an "arm metal" benchmark in the works, not sure whether it will matter much ( not sure how aws provisions metal instances, but it takes them considerably longer to boot, so it may be real hw ). In any case: comparing apples to apples is prudent here, there are a few more branches in that repo ( and more will show up soon ) : click around. |
|
Yeah, I saw thats the r6x machine. I'm thinking it might be feasable to have an off-site (read: AWS) setup that handles the PC1 for me. Probably will have to squeeze PC2 in there, too, because otherwise its going to be hundreds of GB's that need to cross the network. But if a remote setup that costs ~200EUR / month can produce 1 sealed sector (2 if you'd go for the r6x2 with 128GB ram) per 2 hours, and then transfer those 32GB sectors over the internet to a simpler machine - it might very well be worth it.. :) |
|
I installed Archlinux on my Raspberry Pi 4, and managed to compile Lotus on it. However, when running the benchmarks, this is what happened: If I'm not mistaken it originates from here: https://github.com/filecoin-project/filecoin-ffi/blob/master/generated/generated.go#L587 Would that mean that this CPU is missing a certain instructionset? |
|
@RobQuistNL are you using any special flags/envvars when compiling? Is your golang downloaded or built from source? There are some weird stories about TLDR: need more info to help you with this |
|
I installed golang through pacman if I'm not mistaken. But no worries - I don't think it makes any sense to run on a Raspberry anyway... |
|
@RobQuistNL fair, for @deefactorial @LyleLee we are tracking the merging of #2176 (comment) in a different ticket. Aside from that: can this (build) issue be considered resolved? If you experience errors like @RobQuistNL perhaps open new ones so that we can keep the conversation on a specific subtopic? |
It is a fix for lotus. But the fix affects go-filecoin compiling which I can't work it through. |
|
@ribasushi There is a new situation here, I modify the patch a little bit since banana@28b9f781d968:~/lotus/extern/filecoin-ffi$ git diff --cached diff --git a/rust/Cargo.toml b/rust/Cargo.toml
index 648eef4..a4b6092 100644
--- a/rust/Cargo.toml
+++ b/rust/Cargo.toml
@@ -16,7 +16,7 @@ crate-type = ["rlib", "staticlib"]
bls-signatures = "0.6.0"
byteorder = "1.2"
drop_struct_macro_derive = "0.4.0"
-ff = { version = "0.2.1", package = "fff" }
+ff = { version = "0.2.3", package = "fff" }
ffi-toolkit = "0.4.0"
libc = "0.2.58"
log = "0.4.7"
@@ -39,3 +39,6 @@ cbindgen = "= 0.14.0"
[dev-dependencies]
tempfile = "3.0.8"
+[patch.crates-io]
+filecoin-proofs = { git = "https://github.com/filecoin-project/rust-fil-proofs" , branch = "feat/aarch64" }
+fil-sapling-crypto = { git = "https://github.com/filecoin-project/sapling-crypto", branch = "master" }
diff --git a/rust/rust-toolchain b/rust/rust-toolchain
index 3987c47..bf867e0 100644
--- a/rust/rust-toolchain
+++ b/rust/rust-toolchain
@@ -1 +1 @@
-1.43.1
+nightly |
|
@cryptonemo could you please comment on whether a new tag for rust-ffi is upcoming, so I can bump it in lotus properly? It is fine if not planned, just checking what's the best way forward. |
|
@ribasushi We'll be releasing a new |
|
@LyleLee ^^ let's wait for the above and then everything will get fixed for good. Too many moving parts at present. |
@ribasushi @cryptonemo Good job! Looking forward to that. |
|
The |
I test the pull/124, still seeing errors that need help: |
|
@ribasushi The aarch64 platform you were using is not related to Raspberry Pi 64, correct? Does that actual target platform still build? I'm not specifically aware of Raspberry Pi 64 being a supported targetm can you comment there? |
|
He was running on AARCH64 platforms hosted by Amazon - I'm the idiot trying to run it on a raspberry pi :) |
|
@LyleLee @RobQuistNL error on our side, we'll update when ready for retest |
|
@ribasushi set me up with an aarch64 node and I'm unable to get this to compile. Strangely, I'm also unable to get the feat/aarch64 branch building (merged in filecoin-project/rust-fil-proofs#1204). It was reported that at one point this worked, so either 1) I'm missing something about how whoever tested it was building it, or 2) Some dependencies have changed that are now broken since then (unlikely). Anyone have any info on how the original testing was done, or if those setups still work? |
|
@cryptonemo it was me that got it to work previously. I'll get things to build in the next ~6 hours and post exact branch/instructions here. |
I do believe it was working -- as you know, just trying to figure out what's changed or if I'm missing something. Thanks! |
|
@LyleLee @RobQuistNL there has been some movement on this, should be resolved by EoW |
|
@ribasushi Thanks. I setup builds on Travis CI where support both x86 and Arm64. And x86 builds good but fail on Arm64.
The patches on filecoin-proofs and fil-sapling-crypto are still needed, my thought. |
|
@LyleLee errors in the release process needs to be addressed, so that no patches are needed. I can provide you with a recipe on how to build lotus against all |
|
@ribasushi At the last few line of Travis CI job log: Arm64 job failed: https://travis-ci.org/github/LyleLee/lotus/jobs/715412869: Yes, we can manage to fix this and build Lotus master branch by crafting a specific recipe. But it will be great if lotus master naturally builds successfully on Arm64. |
|
@ribasushi I think we're resolved on this now? |
|
@cryptonemo @LyleLee I need to perform one more test on this, will update/close when ready ( EoW ) |
|
@LyleLee AT LAST! On your The only difference from "official compilation" ( hence the required patch ) is that the build system of filecoin-ffi does not honor |
|
@ribasushi Awesome! Verified! Both X86 and aarch64 are now building smooth as silver. |
|
@LyleLee note that lotus |
Describe the bug
When trying to build the lotus project from source files I run into this error:
To Reproduce
Steps to reproduce the behavior:
lotus aarch64 make all error #1779
Expected behavior
The lotus project to build without errors
Screenshots
Someone already reported the issue on an upstream repo.
zcash/sapling-crypto#104
Version (run
lotus --version):unable to compile the latest version
Additional context
Add any other context about the problem here.
Arm64 Architecture
The text was updated successfully, but these errors were encountered: