Deno build on riscv64

[hack3ric]

I've successfully built Deno (1.32.3, 1.32.4 in progress) in qemu-user and run on HiFive Unmatched on Arch Linux riscv64.

Some tweaks I did:

• Use system clang, lld, gn and ninja to manually build V8.
• Apply Support for unconventional builds rusty_v8#1209 to allow passing extra GN args when building V8. The following steps will contain newly added environment variables from that PR.
• NO_PRINT_GN_ARGS=1. GN somehow segfaults when printing arguments, but it is trivial for now and doesn't affect generating build files.
• Enable v8_enable_shared_ro_heap. This is intentially disabled, but we need to wait for pointer compression to meet rusty_v8's requirement and disable it.
• Replace ring with felixonmars/ring to provide C fallback for platforms without assembly crypto implementation.

Environment variables exported:

local _extra_gn_args=(
  'custom_toolchain="//build/toolchain/linux/unbundle:default"'
  'host_toolchain="//build/toolchain/linux/unbundle:default"'
  'v8_enable_shared_ro_heap=true'
)

export CC=clang CXX=clang++ AR=ar NM=nm
export CFLAGS="${CFLAGS//-fstack-clash-protection/}" CXXFLAGS="${CXXFLAGS//-fstack-clash-protection/}"
export V8_FROM_SOURCE=1
export CLANG_BASE_PATH=/usr
export GN=/usr/bin/gn NINJA=/usr/bin/ninja
export EXTRA_GN_ARGS="${_extra_gn_args[@]}"
export NO_PRINT_GN_ARGS=1

Build script for Arch Linux can be found in felixonmars/archriscv-packages#2510. Patch Arch Linux's official PKGBUILD and build it:

pacman -S asp
asp checkout deno
cd deno/repos/community-x86_64
curl -L https://github.com/felixonmars/archriscv-packages/pull/2510.patch | git apply
makepkg -s

Caveats

Note that current build runs TypeScript scripts normally, but in REPL errors will appear (while not affecting basic auto completion functionality):

ERROR TSLS = Error: Could not find source file: 'file://<working_dir>/$deno$repl.ts'.
    at getValidSourceFile (ext:deno_tsc/00_typescript.js:133358:23)
    at Object.getSemanticDiagnostics (ext:deno_tsc/00_typescript.js:133600:32)
    at serverRequest (ext:deno_tsc/99_main_compiler.js:1059:34)
    at [ext:cli/lsp/tsc.rs:3496:26]:1:12

This is triggered only by TypeScript LSP's getDiagnostics request. Other LSP requests with specifiers containing $deno$repl.ts seem to not fail. I'm yet to find out why.

[kxxt]

The errors from TypeScript LSP is perhaps caused by some path manipulation operations in 00_typescript.js.

In function pathForLibFile in 00_typescript.js:

function hexEncode(s) {
  // helper function for debugging
  let result = "";
  for(let i=0; i < s.length; i++) {
    let hex = s.charCodeAt(i).toString(16);
    result += ("000" + hex).slice(-4);
  }
  return result;
}
function pathForLibFile(libFileName) {
  const core = globalThis.Deno.core;
  core.print(`entering pathForLibFile(${libFileName}): \n`);
  const components = libFileName.split(".");
  let path = components[1];
  core.print(`!!!!! path = ${path}, components = ${components}\n`, true);
  let i = 2;
  while (components[i] && components[i] !== "d") {
    let expr = (i === 2 ? "/" : "-") + components[i];
    core.print(`!!!!! path += ${hexEncode(expr)}\n`, true);
    path += expr;
    i++;
  }
  const resolveFrom = combinePaths(currentDirectory, `__lib_node_modules_lookup_${libFileName}__.ts`);
  core.print(`pathForLibFile(${libFileName}): \n    defaultLibraryPath = ${defaultLibraryPath}\n    resolveFrom = ${resolveFrom}\n    path = ${path}\n`, true);
  const localOverrideModuleResult = resolveModuleName("@typescript/lib-" + path, resolveFrom, { moduleResolution: 2 /* Node10 */ }, host, moduleResolutionCache);
  if (localOverrideModuleResult == null ? void 0 : localOverrideModuleResult.resolvedModule) {
    return localOverrideModuleResult.resolvedModule.resolvedFileName;
  }
  return combinePaths(defaultLibraryPath, libFileName);
}

The while loop is producing wrong results on riscv64:

before call processRootFile: lib.deno.ns.d.ts   entering pathForLibFile(lib.deno.ns.d.ts):
!!!!! path = deno, components = lib,deno,ns,d,ts        !!!!! path += 0060006e0073      !!!!! path += 000b0064  !!!!! path += 000b00740073      pathForLibFile(lib.deno.ns.d.ts):
defaultLibraryPath = asset:///lib.esnext.d.ts
resolveFrom = cache:///__lib_node_modules_lookup_lib.deno.ns.d.ts__.ts
path = deno`ns
d
ts   resolveModuleName(@typescript/lib-deno`ns
d
ts, cache:///__lib_node_modules_lookup_lib.deno.ns.d.ts__.ts, [object Object], [object Object], undefined, undefined, undefined)

The log doesn't look nice. There are some extra characters:

for "ns", there is an extra U+0060(`) before it
for "d" and "ts", there is an extra U+000B(Line Tabulation) before it
(The loop actually shouldn't reach here, the correct result is "deno/ns")

It looks like a V8 bug.

By explicitly using char codes I can workaround this bug:

// while (components[i] && components[i] !== "d") {
while (components[i] && components[i].charCodeAt(0) !== 100) {
// let expr = (i === 2 ? "/" : "-") + components[i];
  let expr;
  if (i === 2)
        expr = String.fromCharCode(47) + components[i];
  else
        expr = String.fromCharCode(45) + components[i];

But there are other places that have bugs about path manipulation operations. So only fixing this one place can't let that error go away.

Any suggestions and helps will be well appreciated!

[hack3ric]

Hello, I want to update the status of Deno on riscv64.

On 1.33.1 denoland/rusty_v8#1209 is merged and released. The $deno$repl.ts problem disappeared after disabling v8_enable_shared_ro_heap (as GN parameter restriction was removed in denoland/rusty_v8@07f2e9f). Other parameters are unchanged.

local _extra_gn_args=(
  'custom_toolchain="//build/toolchain/linux/unbundle:default"'
  'host_toolchain="//build/toolchain/linux/unbundle:default"'
)

export CC=clang CXX=clang++ AR=ar NM=nm
export CFLAGS="${CFLAGS//-fstack-clash-protection/}" CXXFLAGS="${CXXFLAGS//-fstack-clash-protection/}"
export V8_FROM_SOURCE=1
export CLANG_BASE_PATH=/usr
export GN=/usr/bin/gn NINJA=/usr/bin/ninja
export EXTRA_GN_ARGS="${_extra_gn_args[@]}"
export NO_PRINT_GN_ARGS=1

As a side note, V8's pointer compression on riscv64 is fixed in https://chromium-review.googlesource.com/c/v8/v8/+/4559353. If the aforementioned restriction is added back, Deno is likely to still work properly.

[tuler]

What is the status of deno on riscv64 now that deno v2 is out? Any effort in progress?

[hack3ric]

What is the status of deno on riscv64 now that deno v2 is out? Any effort in progress?

Pretty much usable AFAIK, at least on recent versions <2. 1.46.0 needs to disable pointer compression like before. Progress are tracked in Arch Linux RISC-V's deno patch against Arch Linux PKGBUILD (if this file does not exist then the original PKGBUILD will build fine.)

As for Deno 2.0, I haven't looked at it yet, but as long as it doesn't have groundbreaking changes in (rusty-)V8, it should build and work with no to little effort of porting.

[shodan8192]

Hello, I'm trying deno-1.46.3-2-riscv64 from archriscv.felixc.at on LicheeRV Nano board with @Fishwaldo's Debian.

On some scripts I get Illegal instruction, for example it can be triggered by :

echo 'import mqtt from "npm:mqtt"' | deno run -

Through gdb I've found that it's caused by vle8.v which I believe is RVV 1.0 instruction. Is my example giving the same error on HiFive Unmatched ?

[hack3ric]

Hi @shodan8192 My guess is that your kernel/SBI on Lichee Pi Nano treats RVV 0.7 in C906 cores as RVV 1.0. Could you try upgrading OpenSBI and see if it works?

[shodan8192]

My system report rv64imafdvcsu ISA .
SiFive U740 doesn't have any vector extension, that's why I've asked if my example works on it. If yes - then I have to disable RVV in kernel, if not, then deno should be compiled targeting cpu without vector extension.

[hack3ric]

This is not statically compiled code, it's generated by V8.

In terms of your question -- yes, it runs fine on Unmatched, and SG2042 (which has its RVV 0.7 hidden so application won't recognize it as RVV anymore)

[shodan8192]

Now that's clear. I didn't knew that V8 make use of vector instructions if they're available, thats why I though they were generated by compiler. So I'll try to disable RVV 0.7 and make things work. Thanks @hack3ric !

[shodan8192]

Sorry for spam, but I can't get it to work on Lichee Pi Nano with 5.10 kernel.

I've removed CONFIG_VECTOR and CONFIG_VECTOR_0_7 from kernel config and set riscv,isa = "rv64imafdc" in dts file then rebuild - and nothing changed. Testing with this code :

#include <sys/auxv.h>
#include <stdio.h>
#include <asm/hwcap.h>

void main() {
  unsigned long hw_cap = getauxval(AT_HWCAP);
  printf("I %s\n", hw_cap & COMPAT_HWCAP_ISA_I ? "detected" : "not found");
  printf("M %s\n", hw_cap & COMPAT_HWCAP_ISA_M ? "detected" : "not found");
  printf("A %s\n", hw_cap & COMPAT_HWCAP_ISA_A ? "detected" : "not found");
  printf("F %s\n", hw_cap & COMPAT_HWCAP_ISA_F ? "detected" : "not found");
  printf("D %s\n", hw_cap & COMPAT_HWCAP_ISA_D ? "detected" : "not found");
  printf("C %s\n", hw_cap & COMPAT_HWCAP_ISA_C ? "detected" : "not found");
  printf("V %s\n", hw_cap & COMPAT_HWCAP_ISA_V ? "detected" : "not found");
}

shows :

I detected
M detected
A detected
F detected
D detected
C detected
V not found

/proc/cpuinfo :

processor	: 0
hart		: 0
isa		: rv64imafdc
mmu		: sv39

Running my example with V8 JIT disabled :

echo 'import mqtt from "npm:mqtt"' | deno run --v8-flags=--jitless -

also crashes with SIGILL, so it seems it's not in V8 generated code. What else I can try to see what happening ?

[hack3ric]

It could be another problem with C906. fence.tso was originally not included in G, and C906 doesn't implement it in hardware. Newer OpenSBI for LicheeRV Nano (and other boards that uses C906 cores) should emulate it. Could you try upgrading that?

p.s. Sorry for not replying the email, since you also asked here later I'll answer here.

[shodan8192]

I'm using OpenSBI already patched for fence.tso emulation, and illegal instruction is still one of RVV.

[hack3ric]

I still cannot reproduce this on LicheeRV Nano running Arch Linux RISC-V (kernel 6.11.3-arch1-1), so I'm out of ideas...

[shodan8192]

Kernel 6.11 have riscv_hwprobe syscall. If it's used to get cpu capabilities, and kernel 5.11 doesn't have it, then it can be erroneously assumed that cpu has RVV.

Where I can find image of Arch Linux you using ?

[hack3ric]

I'm using one of our team's build machine. There's no image right now, but I assume you can flash Debian first, replace the chroot, and then modify extlinux.conf or something so it boots to Arch-managed kernel.

Deno on Arch RISC-V is built against newer version of glibc (2.40 actually), and it may not work under your Debian image. As you mentioned in the email the detection method used in V8 is compile-time.

$ echo '#include <stdio.h>' | gcc -dM -E - | grep GLIBC
#define __GLIBC_USE(F) __GLIBC_USE_ ## F
#define __GLIBC_PREREQ(maj,min) ((__GLIBC__ << 16) + __GLIBC_MINOR__ >= ((maj) << 16) + (min))
#define __GLIBC_USE_IEC_60559_TYPES_EXT 0
#define __GLIBC__ 2
#define __GLIBC_USE_IEC_60559_BFP_EXT 0
#define __GLIBC_USE_DEPRECATED_GETS 0
#define __GLIBC_USE_C23_STRTOL 0
#define __GLIBC_USE_DEPRECATED_SCANF 0
#define __GLIBC_USE_LIB_EXT2 0
#define __GLIBC_USE_IEC_60559_FUNCS_EXT 0
#define __GLIBC_USE_IEC_60559_FUNCS_EXT_C23 0
#define __GLIBC_USE_IEC_60559_BFP_EXT_C23 0
#define __GLIBC_USE_ISOC23 0
#define __GLIBC_MINOR__ 40
#define __GLIBC_USE_IEC_60559_EXT 0

You could try to compile and install a newer version of glibc (>=2.40) on Debian though. But yeah, use other distros' packages is considered a bad idea :(

[shodan8192]

In my Debian image there's the same version of glibc ldd (Debian GLIBC 2.40-3) 2.40 as in your Arch. I think that deno is quite distro-agnostic and should run on any Linux with recent glibc version.

I've excluded V8 as crash cause by disabling JIT, and by src code analysing - in my case V8 doesn't try use of rvv nor fpu, because detection method doesn't cover my case (glibc 2.40 and old kernel without hwprobe)

Actually hwprobe syscall is called triple times before crash and only last of them originating from V8 :

syscall_0x102(0x3fffbf6570, 0x1, 0, 0, 0, 0) = -1 ENOSYS (Function not implemented)
syscall_0x102(0x3fffbf6570, 0x1, 0, 0, 0, 0) = -1 ENOSYS (Function not implemented)
syscall_0x102(0x3fffbeded8, 0x1, 0, 0, 0, 0x2ab59305d0) = -1 ENOSYS (Function not implemented)

I'll investigate it further.

[shodan8192]

OK, so I found the problem. In my example crash occur, when npm tgz package is downloaded and is about to decompress. Deno is using flate2 library for that :

deno/cli/npm/managed/cache/tarball_extract.rs

Line 17 in 2c8a0e7

use flate2::read::GzDecoder;

with zlib-ng C library as the backend, which currently has this issue : zlib-ng/zlib-ng#1705 due to RVV detection method.

On my system, if I spoof kernel version to6.10.4 deno works without crash.