deno: 1.6.3 -> 1.8.0 -> 1.8.1

[06kellyjac]

Motivation for this change

Bump deno to 1.7.0 1.8.0 1.8.1

Note, if you like the cat compile test from previous PRs you now need to compile with --allow-read

./result/bin/deno compile --allow-read --unstable https://deno.land/std@0.84.0/examples/cat.ts

• New: Make it easier to override librusty_v8 so that people can revert or update easier with overrides
• Dropped aarch64-linux as it's not released in rusty_v8 0.16.0 aarch64-linux is back in 0.20.0
  • The arch build of rusty_v8 is failing to build aarch64-linux
    • PR tracking adding aarch64-linux back into CI: re-enable aarch64-unknown-linux builds denoland/rusty_v8#589
• Added aarch64-darwin as it's been manually complied and added to releases since 0.14.0
• Removed fetchSubmodules as it couldn't fetch wpt at detached commit a51d36327febebc8634d7b523a7e620f4ebdbf26. Doesn't seem required for much
  other than testing which we're not running right now.
• Added

---

Things still to resolve:

aarch64-linux missing: I'm not sure what the process is here when dropping an architecture, especially when it should be added back in the next couple of versions. I'm happy to leave this PR in draft until it's fixed. Resolved

Would be nice if someone with an aarch64-darwin machine could give this a go although I'd imagine they're fairly rare among nix users currently and that becomes rarer when you talk about aarch64-darwin + nix + deno users.

New binary denort is used for the new --slim compile option but it can't really use a local denort binary.

result
├── bin
│   ├── deno
│   └── denort
└── share
    ├── bash-completion
    │   └── completions
    │       └── deno.bash
    ├── fish
    │   └── vendor_completions.d
    │       └── deno.fish
    └── zsh
        └── site-functions
            └── _deno

It'll look for a zip matching the target env within the cache (e.g. ~/.cache/deno/dl/release/v1.7.0/denort-x86_64-unknown-linux-gnu.zip) and if it doesn't exist it'll download from https://dl.deno.land

Faffing around trying to make a custom zip so it uses a nix compatible denort will either be impossible or a nightmare. I might look at adding a flag to deno so you can force a specific pre-existing denort binary.
Until then I will probably delete it from the result.

The --slim feature does work though and the result can be ran in a normal distro

λ ./result/bin/deno compile --allow-read --lite --unstable https://deno.land/std@0.79.0/examples/cat.ts
Check https://deno.land/std@0.79.0/examples/cat.ts
Bundle https://deno.land/std@0.79.0/examples/cat.ts
Compile https://deno.land/std@0.79.0/examples/cat.ts
Checking https://dl.deno.land/release/v1.7.0/denort-x86_64-unknown-linux-gnu.zip
Download has been found
Archive:  /tmp/.tmpinJrgq/denort..zip
  inflating: denort
Emit cat
λ mv cat cat-slim
λ ./result/bin/deno compile --allow-read --unstable https://deno.land/std@0.79.0/examples/cat.ts
Check https://deno.land/std@0.79.0/examples/cat.ts
Bundle https://deno.land/std@0.79.0/examples/cat.ts
Compile https://deno.land/std@0.79.0/examples/cat.ts
Emit cat
λ ls -al cat*
.rwxrwxrwx jk users 44 MB Wed Jan 20 11:48:26 2021  cat
.rwxrwxrwx jk users 33 MB Wed Jan 20 11:47:46 2021  cat-slim
λ ls /tmp/.tmpinJrgq/denort..zip
 /tmp/.tmpinJrgq/denort..zip
λ ls /tmp/.tmpinJrgq
 denort   denort..zip
λ ./cat-slim README.md
zsh: no such file or directory: ./cat-slim
λ docker run -v $PWD:/dir -it debian:buster-slim
$ cd dir
$ ./cat-slim README.md
<p align="center">
# ...

I haven't bothered with the new --target option as I've seen some compilation issues in the deno GH issues & when working it'll have a pretty similar result to --lite

New note:

this is the ldd of the --slim build:

./result/bin/deno compile --allow-read --lite --unstable https://deno.land/std@0.79.0/examples/cat.ts
Check https://deno.land/std@0.79.0/examples/cat.ts
Bundle https://deno.land/std@0.79.0/examples/cat.ts
Compile https://deno.land/std@0.79.0/examples/cat.ts
Archive:  /tmp/.tmpFvreLw/denort..zip
  inflating: denort
Emit cat
λ ldd cat
	linux-vdso.so.1 (0x00007ffc27bf6000)
	libdl.so.2 => /nix/store/hp8wcylqr14hrrpqap4wdrwzq092wfln-glibc-2.32-37/lib/libdl.so.2 (0x00007f0c3bf44000)
	libstdc++.so.6 => not found
	libgcc_s.so.1 => /nix/store/hp8wcylqr14hrrpqap4wdrwzq092wfln-glibc-2.32-37/lib/libgcc_s.so.1 (0x00007f0c3bf2a000)
	librt.so.1 => /nix/store/hp8wcylqr14hrrpqap4wdrwzq092wfln-glibc-2.32-37/lib/librt.so.1 (0x00007f0c3bf1f000)
	libpthread.so.0 => /nix/store/hp8wcylqr14hrrpqap4wdrwzq092wfln-glibc-2.32-37/lib/libpthread.so.0 (0x00007f0c3befe000)
	libm.so.6 => /nix/store/hp8wcylqr14hrrpqap4wdrwzq092wfln-glibc-2.32-37/lib/libm.so.6 (0x00007f0c3bdbb000)
	libc.so.6 => /nix/store/hp8wcylqr14hrrpqap4wdrwzq092wfln-glibc-2.32-37/lib/libc.so.6 (0x00007f0c3bbf8000)
	/lib64/ld-linux-x86-64.so.2 => /nix/store/hp8wcylqr14hrrpqap4wdrwzq092wfln-glibc-2.32-37/lib64/ld-linux-x86-64.so.2 (0x00007f0c3e693000)

so there might be a way to make it work 🤷

---

new webGPU API is having some issues seems to be working more consistently now

./result/bin/deno run --unstable --allow-write=output.png https://raw.githubusercontent.com/crowlKats/webgpu-examples/f3b979f57fd471b11a28c5b0c91d0447221ba77b/hello-triangle/mod.ts

error: TS2304 [ERROR]: Cannot find name 'GPUTexture'.
  texture: GPUTexture;
...

But using the prebuilt 1.8.0 release on my arch linux machine I'm getting no suitable adapter found

Update - using 1.8.1:

seems to behave like other machines

$ ./result/bin/deno run --unstable --allow-write=output.png https://raw.githubusercontent.com/crowlKats/webgpu-examples/main/hello-triangle/mod.ts
Download https://raw.githubusercontent.com/crowlKats/webgpu-examples/main/hello-triangle/mod.ts
Download https://raw.githubusercontent.com/crowlKats/webgpu-examples/main/utils.ts
Download https://raw.githubusercontent.com/crowlKats/webgpu-examples/main/deps.ts
Check https://raw.githubusercontent.com/crowlKats/webgpu-examples/main/hello-triangle/mod.ts
no suitable adapter found

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
• Built on platform(s)
  • NixOS (x86_64)
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

[06kellyjac]

I'm probably not going to bump the version to 1.7.1 until rusty_v8 on arm64 is fixed

---

edit: 1.7.2 released, still waiting on aarch64 release of rusty_v8

[06kellyjac]

We're on v1.7.4 and there's still no aarch64 rusty_v8 builds

There are 3 options

• Try fix the CI for aarch64 linux rusty_v8 so it builds
  • not likely but worth a try 🤷
• Try build rusty_v8 from source but using our v8 drv (or prebuilt v8)
  • depends how long the build takes, also there are some packages that are starting to want rusty_v8 too and having many versions of rusty_v8 in the cache might suck
• Drop aarch64 linux 😞

[06kellyjac]

Any thoughts on the options in the comment above? @marsam @frontsideair

[frontsideair]

The most robust solutions seems to be the second one, to build rusty_v8 from source, but I have no idea how much effort would that require.

[06kellyjac]

It's been a while since I tried it so I can't remember but I'll give it another go

[06kellyjac]

So as an update I've learnt that librusty_v8.a is libv8.a compiled with extra bits from src/binding.cc

I've documented the two options, try to build on-top of libv8.a or build v8 + src/binding.cc completely from scratch in #114450

There are a couple of headers missing in version 7 of v8 so using our pre-existing builds is on-hold for now. I'll take this time to try the second option of completely from source, see how hard it is, how long it takes on my machine, etc

[06kellyjac]

Not been much luck building librusty_v8.a I might come back to it at some point in the future but denoland/rusty_v8#628 added aarch64 linux back, https://github.com/denoland/rusty_v8/releases/tag/v0.20.0 has the prebuilt lib, and https://github.com/denoland/deno/releases/tag/v1.8.0 uses v0.20.0 so we're good to go

Just sorting out the update now

[06kellyjac]

Result of nixpkgs-review pr 110156 run on x86_64-linux 1

1 package built:

• deno

[06kellyjac]

Anyone with an aarch64-darwin machine that can test this? 😅

[frontsideair]

@06kellyjac would that be a regular mac? I can do it if so.

[06kellyjac]

@frontsideair the aarch64-darwin machines are the new M1 chip ones but regular mac testing would be great too!

[frontsideair]

I don't have one of those but I got an error during build on my Intel Mac. Here's the tail:

Tail

The following warnings were emitted during compilation:

warning: extern/exception.m:1:10: fatal error: 'objc/objc.h' file not found
warning: #include <objc/objc.h>
warning:          ^~~~~~~~~~~~~
warning: 1 error generated.

error: failed to run custom build command for `objc_exception v0.1.2`

Caused by:
  process didn't exit successfully: `/private/tmp/nix-build-deno-1.8.0.drv-0/source/target/release/build/objc_exception-d2fde7e8ea1dc86e/build-script-build` (exit code: 1)
  --- stdout
  TARGET = Some("x86_64-apple-darwin")
  OPT_LEVEL = Some("0")
  HOST = Some("x86_64-apple-darwin")
  CC_x86_64-apple-darwin = Some("/nix/store/8i0yaj3r82kqcdbr453wzzijnszxg4gx-clang-wrapper-7.1.0/bin/cc")
  CFLAGS_x86_64-apple-darwin = None
  CFLAGS_x86_64_apple_darwin = None
  HOST_CFLAGS = None
  CFLAGS = None
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some("false")
  CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2,sse3,ssse3")
  running: "/nix/store/8i0yaj3r82kqcdbr453wzzijnszxg4gx-clang-wrapper-7.1.0/bin/cc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-arch" "x86_64" "-Wall" "-Wextra" "-o" "/private/tmp/nix-build-deno-1.8.0.drv-0/source/target/release/build/objc_exception-940d5c699681c847/out/extern/exception.o" "-c" "extern/exception.m"
  cargo:warning=extern/exception.m:1:10: fatal error: 'objc/objc.h' file not found
  cargo:warning=#include <objc/objc.h>
  cargo:warning=         ^~~~~~~~~~~~~
  cargo:warning=1 error generated.
  exit code: 1

  --- stderr


  error occurred: Command "/nix/store/8i0yaj3r82kqcdbr453wzzijnszxg4gx-clang-wrapper-7.1.0/bin/cc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-arch" "x86_64" "-Wall" "-Wextra" "-o" "/private/tmp/nix-build-deno-1.8.0.drv-0/source/target/release/build/objc_exception-940d5c699681c847/out/extern/exception.o" "-c" "extern/exception.m" with args "cc" did not execute successfully (status code exit code: 1).


warning: build failed, waiting for other jobs to finish...
The following warnings were emitted during compilation:

warning: extern/exception.m:1:10: fatal error: 'objc/objc.h' file not found
warning: #include <objc/objc.h>
warning:          ^~~~~~~~~~~~~
warning: 1 error generated.

error: build failed
builder for '/nix/store/z3qrjvxyggj6fyzn4jkga2qb5ysb21gp-deno-1.8.0.drv' failed with exit code 101
error: build of '/nix/store/z3qrjvxyggj6fyzn4jkga2qb5ysb21gp-deno-1.8.0.drv' failed

[06kellyjac]

updated to 1.8.1 still need to resolve the mac issue

[marsam]

LGTM. Thank you!

[frontsideair]

Can confirm that it builds fine on my Intel Mac.

[06kellyjac]

Can confirm that it builds fine on my Intel Mac.

Ooh nice. I had a look around for that missing header file but couldn't find anything. Thanks @marsam