Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deno: init at 1.0.0 #87653

Merged
merged 2 commits into from
May 23, 2020
Merged

Deno: init at 1.0.0 #87653

merged 2 commits into from
May 23, 2020

Conversation

06kellyjac
Copy link
Member

@06kellyjac 06kellyjac commented May 12, 2020

Motivation for this change

Adding deno for it's 1.0.0 release

Resolves: #78461
Tracking issue for 1.0.0 release: denoland/deno#2473
PR for 1.0.0 release: denoland/deno#5273
Release post: https://deno.land/v1

Version is currently 1.0.0-rc3 as that's the latest release. I will force-push once 1.0.0 is out (ETA 13th May)

Any and all suggestions and criticisms welcome, most of my nix packaging experience is on my own machine so I'd like to know if I'm straying from best/good practices.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS (x86_64)
    • macOS
    • NixOS or other (aarch64 ARM)
    • Build passed, Build + Check ended SIGKILL 9 (Raspbian RPI4)
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

Concerns

  • Currently using built zip Now building from source, not rebuilding v8 though
    • Probably preferable to build from source releases but had issues with rusty_v8 Solved
    • Building from source + also building v8 from source is probably not preferred as v8 takes ages to build (maybe have it as optional?)
  • The release has a .d.ts definition
    • I haven't done much actual work with deno yet so IDK if I should install this somewhere
    • The deno_install script doesn't seem to touch it so it should be fine
    • Can just grab this from running deno tags > lib.deno.d.ts
  • It has it's own upgrade command
    • Just hope people dont touch it?/Let it not work?
    • Add a patch to remove or hide it?
  • No versioning like existing node (and other language/interpreter) releases
    • Not of particular concern now but something to think about for future versions

Tests of deno after install:

deno --version
deno 1.0.0
v8 8.4.300
typescript 3.9.2

Examples from help

deno run https://deno.land/std/examples/welcome.ts
Welcome to Deno

deno eval "console.log(30933 + 404)"
31337

Watch fs

watch.ts:

const watcher = Deno.watchFs("./");
for await (const event of watcher) {
  console.log(">>>> event", event);
  // { kind: "create", paths: [ "/foo.txt" ] }
}

deno run --allow-read watch.ts

Deno Packages/Programs

deno install --unstable --allow-read --allow-run -f https://deno.land/x/denon/denon.ts

~/.deno/bin/denon log.ts

log.ts:

console.log("hello, world!")

@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux labels May 12, 2020
@06kellyjac
Copy link
Member Author

Force-pushed 1.0.0-rc3

@06kellyjac
Copy link
Member Author

06kellyjac commented May 12, 2020

I've finally got past rusty_v8 while building deno with rust. Needed to include submodules, rebuilding now 🤞


Edit:
After a bit of tweaking it worked. Now doing some final cleanup.

@06kellyjac
Copy link
Member Author

Force-pushed move to building from source, rusty_v8 is still using the .a lib file from it's release, not building the whole of v8 from source.

pkgs/development/web/deno/default.nix Outdated Show resolved Hide resolved
pkgs/development/web/deno/default.nix Outdated Show resolved Hide resolved
@06kellyjac 06kellyjac force-pushed the deno_release branch 2 times, most recently from 574aec7 to ad637a1 Compare May 13, 2020 00:52
@ofborg ofborg bot added 8.has: package (new) This PR adds a new package 11.by: package-maintainer This PR was created by the maintainer of the package it changes 10.rebuild-darwin: 1-10 10.rebuild-darwin: 1 10.rebuild-linux: 1-10 10.rebuild-linux: 1 and removed 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux labels May 13, 2020
@06kellyjac
Copy link
Member Author

Force-pushed completions for:

  • bash
  • zsh
  • fish

Need to test them with zsh and fish but should work fine

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/deno-an-expression-in-need-of-love/6978/5

@06kellyjac
Copy link
Member Author

Deno 1.0.0 release pr: denoland/deno#5273

@06kellyjac
Copy link
Member Author

aarch64-linux & x86_64-darwin were the wrong way around

@06kellyjac
Copy link
Member Author

06kellyjac commented May 13, 2020

Built 1.0.0-rc3 on a raspberry pi4 aarm64. Failed compiling the debug release for testing...

[~/projects/nixpkgs]$ uname -rmo
4.19.113-v8+ aarch64 GNU/Linux

[~/projects/nixpkgs]$ git status
On branch deno_release
Your branch is up to date with 'origin/deno_release'.

nothing to commit, working tree clean
[~/projects/nixpkgs]$ nix-env -f . -iA deno
...
Finished release [optimized] target(s) in 45m 19s
running tests
...
   Compiling dprint-plugin-typescript v0.17.2
error: could not compile `deno`.

Caused by:
  process didn't exit successfully: `rustc --crate-name deno --edition=2018 cli/main.rs --error-format=json --json=diagnostic-rendered-ansi --emit=dep-info,link -C debuginfo=2 --test -C metadata=f817e861d9945b11 -C extra-filename=-f817e861d9945b11 --out-dir /build/source/target/debug/deps -C linker=/nix/store/9niqfb3kk208ys9ydlq0fwbw98an0106-gcc-wrapper-9.3.0/bin/cc -C incremental=/build/source/target/debug/incremental -L dependency=/build/source/target/debug/deps --extern atty=/build/source/target/debug/deps/libatty-fef2dfa6c65b2b05.rlib --extern base64=/build/source/target/debug/deps/libbase64-b66c018463dbdc5d.rlib --extern byteorder=/build/source/target/debug/deps/libbyteorder-14946c13985e5c43.rlib --extern bytes=/build/source/target/debug/deps/libbytes-3a19f65365565239.rlib --extern clap=/build/source/target/debug/deps/libclap-fab552898178d016.rlib --extern deno_core=/build/source/target/debug/deps/libdeno_core-eec10ff8e7186357.rlib --extern deno_typescript=/build/source/target/debug/deps/libdeno_typescript-f3d5e548f31ddf87.rlib --extern dirs=/build/source/target/debug/deps/libdirs-1a2763926d6b4555.rlib --extern dlopen=/build/source/target/debug/deps/libdlopen-8b686741c2fc7c9c.rlib --extern dprint_plugin_typescript=/build/source/target/debug/deps/libdprint_plugin_typescript-727af653de5d9a20.rlib --extern futures=/build/source/target/debug/deps/libfutures-8cc9414c094d8fe9.rlib --extern glob=/build/source/target/debug/deps/libglob-fb75bf7343f74590.rlib --extern http=/build/source/target/debug/deps/libhttp-d9f8e41936a099ae.rlib --extern indexmap=/build/source/target/debug/deps/libindexmap-8fd98f84d65052fd.rlib --extern lazy_static=/build/source/target/debug/deps/liblazy_static-78a882cda67724e7.rlib --extern libc=/build/source/target/debug/deps/liblibc-e9541b678dbfad84.rlib --extern log=/build/source/target/debug/deps/liblog-55477e12cb4993f5.rlib --extern nix=/build/source/target/debug/deps/libnix-cee17f0a5c51c5ed.rlib --extern notify=/build/source/target/debug/deps/libnotify-632450b49db22219.rlib --extern os_pipe=/build/source/target/debug/deps/libos_pipe-824e577dd9b8e596.rlib --extern pty=/build/source/target/debug/deps/libpty-08ab54bbad4668dc.rlib --extern rand=/build/source/target/debug/deps/librand-adbbdab05878a290.rlib --extern regex=/build/source/target/debug/deps/libregex-41f9a317bbe47bfc.rlib --extern reqwest=/build/source/target/debug/deps/libreqwest-0b9660a552e603e9.rlib --extern ring=/build/source/target/debug/deps/libring-4f9ee9a4e08abf4f.rlib --extern rustyline=/build/source/target/debug/deps/librustyline-03df85814a954098.rlib --extern semver_parser=/build/source/target/debug/deps/libsemver_parser-c4ac13c406950709.rlib --extern serde=/build/source/target/debug/deps/libserde-6c179e6aa0cf0be6.rlib --extern serde_derive=/build/source/target/debug/deps/libserde_derive-a375412b8641c0bb.so --extern serde_json=/build/source/target/debug/deps/libserde_json-681390485c994dd9.rlib --extern sourcemap=/build/source/target/debug/deps/libsourcemap-565dac8c2f7bce31.rlib --extern swc_ecma_visit=/build/source/target/debug/deps/libswc_ecma_visit-8a4cacbd2e599034.rlib --extern sys_info=/build/source/target/debug/deps/libsys_info-31bc2aa900a92b31.rlib --extern tempfile=/build/source/target/debug/deps/libtempfile-715d22f8d7133d77.rlib --extern termcolor=/build/source/target/debug/deps/libtermcolor-8ef38709a6a99f87.rlib --extern tokio=/build/source/target/debug/deps/libtokio-e0455fd5c68b48f4.rlib --extern tokio_rustls=/build/source/target/debug/deps/libtokio_rustls-c2ed0e4cf528b57d.rlib --extern tokio_tungstenite=/build/source/target/debug/deps/libtokio_tungstenite-94eb62287c55047c.rlib --extern url=/build/source/target/debug/deps/liburl-3bb84cd0761910fc.rlib --extern utime=/build/source/target/debug/deps/libutime-dbeedda2ebc15673.rlib --extern uuid=/build/source/target/debug/deps/libuuid-3de8d11cd89a3839.rlib --extern walkdir=/build/source/target/debug/deps/libwalkdir-cf93ac4e794571bd.rlib --extern warp=/build/source/target/debug/deps/libwarp-1ad00a034b0d9e46.rlib --extern webpki=/build/source/target/debug/deps/libwebpki-2139cec367d30e34.rlib --extern webpki_roots=/build/source/target/debug/deps/libwebpki_roots-14888243df210182.rlib -L /build/source/target/debug/gn_out/obj -L native=/build/source/target/debug/build/ring-4e45ae99a72a8bd2/out -L native=/build/source/target/debug/build/sys-info-5d538328efc1bf0e/out` (signal: 9, SIGKILL: kill)
warning: build failed, waiting for other jobs to finish...
error: build failed
builder for '/nix/store/5jvzz27wsgi6ky6g3vj8qm86njbd8zm2-deno-1.0.0-rc3.drv' failed with exit code 101
error: build of '/nix/store/5jvzz27wsgi6ky6g3vj8qm86njbd8zm2-deno-1.0.0-rc3.drv' failed

(signal: 9, SIGKILL: kill) is pretty odd.
I'll try build again without the tests.

Probably best not to build compile heavy programs directly on the arm device. I need to try compile for aarm64 on my x86_64 machine but my libvirt doesn't show UEFI mode...


Without checks works fine.

   Compiling dprint-plugin-typescript v0.17.2
    Finished release [optimized] target(s) in 45m 06s
installing
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/ncd9m5sb8n3k3vmvq8nrbyssyllzgcsj-deno-1.0.0-rc3
shrinking /nix/store/ncd9m5sb8n3k3vmvq8nrbyssyllzgcsj-deno-1.0.0-rc3/lib/libtest_plugin.so
shrinking /nix/store/ncd9m5sb8n3k3vmvq8nrbyssyllzgcsj-deno-1.0.0-rc3/bin/hyper_hello
shrinking /nix/store/ncd9m5sb8n3k3vmvq8nrbyssyllzgcsj-deno-1.0.0-rc3/bin/deno
strip is /nix/store/rkj5n3p7v7mff4j4n6f2nld7mpfb550r-binutils-2.31.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/ncd9m5sb8n3k3vmvq8nrbyssyllzgcsj-deno-1.0.0-rc3/lib  /nix/store/ncd9m5sb8n3k3vmvq8nrbyssyllzgcsj-deno-1.0.0-rc3/bin
patching script interpreter paths in /nix/store/ncd9m5sb8n3k3vmvq8nrbyssyllzgcsj-deno-1.0.0-rc3
checking for references to /build/ in /nix/store/ncd9m5sb8n3k3vmvq8nrbyssyllzgcsj-deno-1.0.0-rc3...
building '/nix/store/n80ma4ch0cps0xs3h55bqr824p4hy0jk-user-environment.drv'...
created 39 symlinks in user environment

Maybe just a timeout issue? Could also disable checks if not x86_64-linux

@doronbehar
Copy link
Contributor

Hey all.

I have absolutely no idea what deno is, but there's this someone who I've talked with at discourse who have made interesting progress regarding packaging deno while building from source:

https://discourse.nixos.org/t/deno-an-expression-in-need-of-love/6978

I think you'll be able to pick up something from there, while perhaps incorporating some of the advices I gave him.

The thing is, they seem to not want to open a GitHub account so they are not able to put their ideas directly here. I have no idea why but I don't judge. I'm happy to proxy his comments because they seem so nice 💚 .

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/deno-an-expression-in-need-of-love/6978/6

@06kellyjac
Copy link
Member Author

06kellyjac commented May 13, 2020

Hi @doronbehar, thanks! I had a bit of a scan through the discourse when I was first stuck on rusty_v8 being annoying & I was going to read more indepth if I couldn't get rusty_v8 to compile with it's static .a lib.

From my experience playing with building deno & from what I know about v8 is it's a pita to build from source. This build is already around 12mins long on my i7-8850H (12) @ 4.3GHz (need to time it) and 45mins long on a Raspberry Pi 4 (4) @ 1.5GHz (see above.

As far as I can tell the node packages also don't build v8 from source:
nodejs.nix common
v14.nix version & sha for v0.14.2
node-v14.2.0.tar.xz it downloads
hydra build of node v0.14.2
raw log from build of node v0.14.2

tools/make-v8.sh doesn't seem to be called in the logs (other than it's shebang being replaced automatically)
And no mention of V8_BUILD_OPTIONS, cd deps/v8, d8, v8_target_cpu=, out.gn from that script.
(some of the eyecatching parts of those scripts)

hydra build for v8
raw log from build of v8

Also note the node build was "only" "1h 22m 21s" but just v8 on it's own is "2h 29m 29s"!!!


I could be wrong, there could be a load of tests in the v8 run that aren't needed for node etc. but this seems like more than enough depth for the build. Most package managers will just take the bin from the release and you're done. (Kinda like my first iteration)

I'll make a discourse account some time soon to make contacting me easier 🙂

@06kellyjac 06kellyjac marked this pull request as ready for review May 14, 2020 03:41
Copy link
Contributor

@andersk andersk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to CONTRIBUTING.md, the commit message should be “deno: init at 1.0.0”. Otherwise looks good.

@06kellyjac 06kellyjac force-pushed the deno_release branch 2 times, most recently from 718d0d5 to ac3a3df Compare May 14, 2020 05:51
Copy link
Member

@prusnak prusnak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left couple of nits, but they are not blocking the merge.

pkgs/development/web/deno/default.nix Outdated Show resolved Hide resolved
pkgs/development/web/deno/default.nix Outdated Show resolved Hide resolved
@06kellyjac
Copy link
Member Author

06kellyjac commented May 14, 2020

Left couple of nits, but they are not blocking the merge.

Please do add them as comments.

Either it'll merge as is now but I'll know what to fix for later, or I'll get it changed before it merges.
I'm happy either way.

Ignore this - Couldn't see the comments on the new GH App

@06kellyjac
Copy link
Member Author

06kellyjac commented May 14, 2020

Not sure why that run_v8_help test failed on aarm64...
The regex seems right, the output has d8 ...

[cli/tests/integration_tests.rs:2773] part = "\nSynopsis:\n"
[cli/tests/integration_tests.rs:2773] i = 1
[cli/tests/integration_tests.rs:2782] "exit false " = "exit false "
[cli/tests/integration_tests.rs:2782] i = 1
OUTPUT
Synopsis:
  shell [options] [--shell] [<file>...]
  d8 [options] [-e <string>] [--shell] [[--module] <file>...]
  -e        execute a string in V8
...
OUTPUT
EXPECTED
[WILDCARD]
Synopsis:
[WILDCARD]d8[WILDCARD]
EXPECTED
thread 'run_v8_help' panicked at 'pattern match failed', cli/tests/integration_tests.rs:2740:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

tracking issue denoland/deno#5324
tracking pr denoland/deno#5402

@Gaelan
Copy link
Contributor

Gaelan commented May 15, 2020

Some small changes to make this build on Darwin:

diff --git a/pkgs/development/web/deno/default.nix b/pkgs/development/web/deno/default.nix
index 8cd2421800b..eb2669cd8b7 100644
--- a/pkgs/development/web/deno/default.nix
+++ b/pkgs/development/web/deno/default.nix
@@ -5,6 +5,9 @@
 , rustPlatform
 , python27
 , installShellFiles
+, Security
+, CoreServices
+, lib
 }:
 let
   pname = "deno";
@@ -48,6 +51,8 @@ rustPlatform.buildRustPackage rec {
     installShellFiles
   ];
 
+  buildInputs = lib.optionals stdenv.isDarwin [ Security CoreServices ];
+
   # The rusty_v8 package will try to download a `librusty_v8.a` release at build time to our read-only filesystem
   # To avoid this we pre-download the file and place it in the locations it will require it in advance
   preBuild = ''
diff --git a/pkgs/top-level/all-packages.nix b/pkgs/top-level/all-packages.nix
index db3491be140..b34d55f5f2f 100644
--- a/pkgs/top-level/all-packages.nix
+++ b/pkgs/top-level/all-packages.nix
@@ -2829,7 +2829,9 @@ in
 
   deer = callPackage ../shells/zsh/zsh-deer { };
 
-  deno = callPackage ../development/web/deno { };
+  deno = callPackage ../development/web/deno {
+    inherit (darwin.apple_sdk.frameworks) Security CoreServices;
+  };
 
   detox = callPackage ../tools/misc/detox { };
 

@prusnak
Copy link
Member

prusnak commented May 15, 2020

No need to add lib, just use stdenv.lib.optionals

@06kellyjac
Copy link
Member Author

Thanks both. I'll get that patched in soon & tested on a mac

@marsam
Copy link
Contributor

marsam commented May 19, 2020

@GrahamcOfBorg build deno

Adding deno for it's 1.0.0 release

* Builds from source (doesn't build V8 from source)
* Runs most of the tests
* Bundles completions for:
  * bash - tested
  * zsh  - untested
  * fish - untested
* Set up for multiple builds:
  * x86_64-linux
    * Passed - NixOS
  * aarch64-linux
    * Passed just build - Raspbian on RPI4
    * Failed build + check (ended with SIGKILL 9)
  * x86_64-darwin
    * Untested
@06kellyjac
Copy link
Member Author

06kellyjac commented May 20, 2020

Deno 1.0.1 launched an hour ago
Should I init at 1.0.1 or merge this & version bump after?


I'll run a build on my machine either way


New sha256: 1g866nffi7mciszqxzmzm5l671lq7in5ziz6412ld2bjwxa7na8l
New cargoSha256: 0584vn3737a1s4c3cz2cg9d214ca0fsqizhyldr462yc5m9lpm9q
rustyv8 is still the same version so no changes there, built fine on x86_64 nixos

@Sorixelle
Copy link
Member

Just a heads up - there's a 1.0.2 now :P Also, re the .d.ts - it's accessible just by running deno types, so there's no need to worry about packaging that.

@06kellyjac
Copy link
Member Author

@Sorixelle correct 👍

Yeah I noticed that when reading the deno CI trying to add an aarch64 release for non-nix users


deno 1.0.2
New sha256: 1sq6a4lph45s6sw4mjyv6p9y60cgfgq98jsyrp81kh32fs0xcqq7
New cargoSha256: 0414s377iqa6rdlgc2d9rjhxbcsjzz0qcp9rva5ykxdw9xbsxv6y
rustyv8 is still the same version so no changes there, built fine on x86_64 nixos

@marsam
Copy link
Contributor

marsam commented May 23, 2020

LGTM, thank you!

@marsam marsam merged commit 0958a7b into NixOS:master May 23, 2020
@06kellyjac
Copy link
Member Author

Thanks @marsam !

Now I'll add a pr for the 1.0.2 version bump :)

@06kellyjac 06kellyjac deleted the deno_release branch May 23, 2020 12:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Deno
8 participants