Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow specifying the path to the sassc binary at build time #91

Closed
wants to merge 1 commit into from

Conversation

anna328p
Copy link

@anna328p anna328p commented Jan 5, 2023

No description provided.

@ntkme
Copy link
Member

ntkme commented Jan 5, 2023

You can already override the binary to be used. You just need to create a tarball that follow the same layout as official release:

bundle config build.sass-embedded SASS_EMBEDDED=/path/to/1.57.1/sass_embedded-1.57.1-linux-x64.tar.gz
bundle install

@ntkme
Copy link
Member

ntkme commented Jan 5, 2023

And sass-embedded has nothing to do with already deprecated sassc.

@ntkme ntkme closed this Jan 5, 2023
@anna328p
Copy link
Author

anna328p commented Jan 5, 2023

You can already override the binary to be used. You just need to create a tarball that follow the same layout as official release:

bundle config build.sass-embedded SASS_EMBEDDED=/path/to/1.57.1/sass_embedded-1.57.1-linux-x64.tar.gz
bundle install

This is not adequate, I would like to be able to point to a binary that the gem doesn't vendor.

I didn't know sassc was a different project, my apologies.

@ntkme
Copy link
Member

ntkme commented Jan 5, 2023

This is not adequate, I would like to be able to point to a binary that the gem doesn't vendor.

I don’t understand what you mean. You just need to pack your own dart-sass-embedded binary into a tarball with the same layout as official dart-sass-embedded release. Then you can supply the your own tarball.

@anna328p
Copy link
Author

anna328p commented Jan 5, 2023

I would like an option to set Sass::Embedded::Compiler::PATH to an arbitrary string. I'm trying to use sass-embedded-host-ruby on NixOS, where you can't run unpatched binaries easily. The sass_embedded binary is not built in a way that would be conducive to packing into a tarball.

@ntkme
Copy link
Member

ntkme commented Jan 5, 2023

Well, technically you can pack a symlink to the actual binary or a shell script that launches the actual binary into a tarball if your own binary is somehow not relocatable.

@anna328p
Copy link
Author

anna328p commented Jan 5, 2023

Why is it necessary that a tarball is provided? From my understanding of this project's Rakefile, you could easily make fetching and unpacking it conditional on a binary path not being provided. (willing to PR this)

@ntkme
Copy link
Member

ntkme commented Jan 5, 2023

Because the release of the host gem is version matched with dart-sass-embedded compiler, the internalization of the dart-sass-embeded binary makes sure it is always correctly paired, and stays as is so that users won’t be able to accidental change the installed binary in anyway. E.g. Imagine that a user would install dart-sass-embedded 1.53.0 from homebrew, and then install matching version of sass-embedded gem 1.53.0, pointing to the homebrew provided dart-sass-embedded externally, at a later point user may update homebrew installation to 1.57.1 but the gem is still 1.53.0, and thus become broken.

In other words, I want the gem installation to be immutable so that I don’t have do deal with user breaking the gem themselves.

@anna328p
Copy link
Author

anna328p commented Jan 5, 2023

Wouldn't providing a tarball with a symlink inside be able to break version constraint guarantees in the same way?

@ntkme
Copy link
Member

ntkme commented Jan 5, 2023

Wouldn't providing a tarball with a symlink inside be able to break version constraint guarantees in the same way?

Yes it will, but that is the last resort only if your binary is not relocatable. As far as I know, Dart VM executable is always relocatable…

@anna328p
Copy link
Author

anna328p commented Jan 5, 2023

Binaries built by Nix link to libraries in immutable locations specific to the hashes of their build dependencies. If the binary is copied into a gem's directory, then the libraries are updated, their new builds will be at a completely different path. When the old ones get garbage-collected, the binary will no longer be able to load its dependencies and so will not run. The gem will need to be rebuilt. This has the potential of happening every time a change is made to the system configuration.

I could create a package that depends on embedded_sass and outputs a tarball containing a single symlink pointing at its location, and set BUNDLE_BUILD__SASS___EMBEDDED='SASS_EMBEDDED=${the path of this package's output}' but that seems like a really dirty workaround...

@ntkme
Copy link
Member

ntkme commented Jan 5, 2023

$ ldd dart-sass-embedded 
        linux-vdso.so.1 (0x0000ffff8a9f1000)
        libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000ffff8a354000)
        libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000ffff8a323000)
        libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000ffff8a278000)
        libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000ffff8a103000)
        /lib/ld-linux-aarch64.so.1 (0x0000ffff8a9c1000)

So the dependency of dart runtime is pretty much just glibc + libpthread, which is very unlikely to change unless you have a major distribution update, so I think relocation should be safe even under Nix.

@ntkme
Copy link
Member

ntkme commented Jan 5, 2023

Maybe you can try patchelf --set-interpreter on the official release and see if simply use Nix’s own ld-linux would make official release work? If that works we can potentially add a step that if we detect NixOS we just patch the binary on the fly during installation with patchelf.

@anna328p
Copy link
Author

anna328p commented Jan 6, 2023

It seems like the dart-sass-embedded executable is... oddly fragile?

tried on a github codespace:

$ ./dart-sass-embedded --version
{
  "protocolVersion": "1.2.0",
  "compilerVersion": "1.57.1",
  "implementationVersion": "1.57.1",
  "implementationName": "Dart Sass",
  "id": 0
}
$ /lib64/ld-linux-x86-64.so.2 ./dart-sass-embedded --version
Dart SDK version: 2.18.6 (stable) (Tue Dec 13 21:15:14 2022 +0000) on "linux_x64"

(this seems to behave like a copy of the dart interpreter binary)

same thing happens when i use patchelf on the binary, even if it looks like a no-op:

$ patchelf --print-interpreter ./dart-sass-embedded 
/lib64/ld-linux-x86-64.so.2
$ patchelf --set-interpreter /lib64/ld-linux-x86-64.so.2 ./dart-sass-embedded 
$ ./dart-sass-embedded 
Usage: dart [<vm-flags>] <dart-script-file> [<script-arguments>]

Executes the Dart script <dart-script-file> with the given list of <script-arguments>.

@ntkme
Copy link
Member

ntkme commented Jan 6, 2023

Looks like we finally know how to reproduce sass/dart-sass-embedded#78, and yes it is a dart runtime issue.

@anna328p
Copy link
Author

anna328p commented Jan 6, 2023

[nix-shell:~/work/ap5.dev]$ ruby -e "puts File.readlink('/proc/self/exe')"
/nix/store/7ca3yj685gdxkf7v5briil6mplmjswki-ruby-3.1.2/bin/ruby

[nix-shell:~/work/ap5.dev]$ ldd $(which ruby)
        linux-vdso.so.1 (0x00007ffda3d97000)
        libruby-3.1.2.so.3.1 => /nix/store/7ca3yj685gdxkf7v5briil6mplmjswki-ruby-3.1.2/lib/libruby-3.1.2.so.3.1 (0x00007fb1ea400000)
        libz.so.1 => /nix/store/lal84wf8mcz48srgfshj4ns1yadj1acs-zlib-1.2.13/lib/libz.so.1 (0x00007fb1ea894000)
        libpthread.so.0 => /nix/store/hsk71z8admvgykn7vzjy11dfnar9f4r1-glibc-2.35-163/lib/libpthread.so.0 (0x00007fb1ea88f000)
        librt.so.1 => /nix/store/hsk71z8admvgykn7vzjy11dfnar9f4r1-glibc-2.35-163/lib/librt.so.1 (0x00007fb1ea88a000)
        libdl.so.2 => /nix/store/hsk71z8admvgykn7vzjy11dfnar9f4r1-glibc-2.35-163/lib/libdl.so.2 (0x00007fb1ea885000)
        libm.so.6 => /nix/store/hsk71z8admvgykn7vzjy11dfnar9f4r1-glibc-2.35-163/lib/libm.so.6 (0x00007fb1ea320000)
        libc.so.6 => /nix/store/hsk71z8admvgykn7vzjy11dfnar9f4r1-glibc-2.35-163/lib/libc.so.6 (0x00007fb1ea000000)
        /nix/store/hsk71z8admvgykn7vzjy11dfnar9f4r1-glibc-2.35-163/lib/ld-linux-x86-64.so.2 => /nix/store/hsk71z8admvgykn7vzjy11dfnar9f4r1-glibc-2.35-163/lib64/ld-linux-x86-64.so.2 (0x00007fb1ea8b4000)

[nix-shell:~/work/ap5.dev]$ /nix/store/hsk71z8admvgykn7vzjy11dfnar9f4r1-glibc-2.35-163/lib/ld-linux-x86-64.so.2 $(which ruby) -e "puts File.readlink('/proc/self/exe')"
/nix/store/hsk71z8admvgykn7vzjy11dfnar9f4r1-glibc-2.35-163/lib/ld-linux-x86-64.so.2

@anna328p
Copy link
Author

anna328p commented Jan 6, 2023

what the fuck

parse_dart_standalone.rb

#!/usr/bin/env ruby

appjit_magic_number = [0xdc, 0xdc, 0xf6, 0xf6, 0, 0, 0, 0]

f = File.read(ARGV[0])

offset, *magic = f.byteslice(-16..).unpack('Q<C*')

if magic == appjit_magic_number
  puts "Magic number: matches"
else
  magic_bytes = magic.map { sprintf "%#x", _1 }.join(', ')
  puts "Magic number: mismatch (#{magic_bytes})"
end

printf "Offset: %i (%#x)\n", offset, offset
[nix-shell:/tmp/tmp.Gv3mO5RrEb]$ ruby parse_dart_standalone.rb dart-sass-embedded 
Magic number: matches
Offset: 5046272 (0x4d0000)

[nix-shell:/tmp/tmp.Gv3mO5RrEb]$ cp dart-sass-embedded dart-sass-embedded.1

[nix-shell:/tmp/tmp.Gv3mO5RrEb]$ patchelf --set-interpreter blarg dart-sass-embedded.1 

[nix-shell:/tmp/tmp.Gv3mO5RrEb]$ ruby parse_dart_standalone.rb dart-sass-embedded.1
Magic number: mismatch (0x6, 0, 0, 0, 0x20, 0, 0, 0)
Offset: 8589934592 (0x200000000)

@anna328p
Copy link
Author

anna328p commented Jan 6, 2023

[nix-shell:/tmp/tmp.Gv3mO5RrEb]$ ls -l
total 17316
-rwxr-xr-x 1 anna users 8845664 Jan  5 20:59 dart-sass-embedded
-rwxr-xr-x 1 anna users 8879440 Jan  5 21:09 dart-sass-embedded.1
-rw-r--r-- 1 anna users     382 Jan  5 21:08 parse_dart_standalone.rb

@anna328p
Copy link
Author

anna328p commented Jan 6, 2023

considering writing a script to separate the file into the runtime and snapshot, patch the runtime, and re-append the snapshot to the patched executable

@anna328p
Copy link
Author

anna328p commented Jan 6, 2023

i just wanted @use in my jekyll scss :(

@anna328p
Copy link
Author

anna328p commented Jan 6, 2023

Loading failed: File offset must be page-aligned.

almost there...

@anna328p
Copy link
Author

anna328p commented Jan 6, 2023

$ ./dse.patched --version
{
  "protocolVersion": "1.2.0",
  "compilerVersion": "1.57.1",
  "implementationVersion": "1.57.1",
  "implementationName": "Dart Sass",
  "id": 0
} 

it works!!!

@anna328p
Copy link
Author

anna328p commented Jan 6, 2023

[nix-shell:/tmp/tmp.nNp4tBoDzI]$ ./dart-sass-embedded 
bash: ./dart-sass-embedded: No such file or directory

[nix-shell:/tmp/tmp.nNp4tBoDzI]$ ldd ./dart-sass-embedded 
        linux-vdso.so.1 (0x00007fff7fd5a000)
        libdl.so.2 => /nix/store/hsk71z8admvgykn7vzjy11dfnar9f4r1-glibc-2.35-163/lib/libdl.so.2 (0x00007fb5a8ab7000)
        libpthread.so.0 => /nix/store/hsk71z8admvgykn7vzjy11dfnar9f4r1-glibc-2.35-163/lib/libpthread.so.0 (0x00007fb5a8ab2000)
        libm.so.6 => /nix/store/hsk71z8admvgykn7vzjy11dfnar9f4r1-glibc-2.35-163/lib/libm.so.6 (0x00007fb5a89d2000)
        libc.so.6 => /nix/store/hsk71z8admvgykn7vzjy11dfnar9f4r1-glibc-2.35-163/lib/libc.so.6 (0x00007fb5a8600000)
        /lib64/ld-linux-x86-64.so.2 => /nix/store/hsk71z8admvgykn7vzjy11dfnar9f4r1-glibc-2.35-163/lib64/ld-linux-x86-64.so.2 (0x00007fb5a9145000)

[nix-shell:/tmp/tmp.nNp4tBoDzI]$ ruby dart_standalone_tool.rb patch-interpreter ./dart-sass-embedded /nix/store/hsk71z8admvgykn7vzjy11dfnar9f4r1-glibc-2.35-163/lib/ld-linux-x86-64.so.2
Magic number: matches
Offset: 5046272 (0x4d0000)
Writing: runtime, /run/user/1000/tmp.kxnxECyVVs/rt.elf (5046272 bytes)
Writing: snapshot, /run/user/1000/tmp.kxnxECyVVs/snap.elf (3799376 bytes)
Running: patchelf --set-interpreter /nix/store/hsk71z8admvgykn7vzjy11dfnar9f4r1-glibc-2.35-163/lib/ld-linux-x86-64.so.2 /run/user/1000/tmp.kxnxECyVVs/rt.elf
Reading: runtime, /run/user/1000/tmp.kxnxECyVVs/rt.elf (5078432 bytes)
Reading: snapshot, /run/user/1000/tmp.kxnxECyVVs/snap.elf (3799376 bytes)
Writing: output, ./dart-sass-embedded
Wrote: runtime (size: 5078432 bytes)
Wrote: padding (size: 608 bytes)
Wrote: snapshot (size: 3799376 bytes)
Wrote: footer
Magic number: matches
Offset: 5079040 (0x4d8000)

[nix-shell:/tmp/tmp.nNp4tBoDzI]$ ./dart-sass-embedded --version
{
  "protocolVersion": "1.2.0",
  "compilerVersion": "1.57.1",
  "implementationVersion": "1.57.1",
  "implementationName": "Dart Sass",
  "id": 0
}

@ntkme
Copy link
Member

ntkme commented Jan 6, 2023

i just wanted @use in my jekyll scss :(

Glad that you are able to get it working! Interestingly, this was actually the reason I started this project.

Eventually this is something that https://github.com/dart-lang/sdk should fix. However, given that the sdk fix probably won’t land in short term, I’m more than happen to include your patch script until upstream fixes the problem. Please free feel to send a PR for adding the script and I’m happy to work on the integration.

@ntkme
Copy link
Member

ntkme commented Jan 6, 2023

dart-lang/sdk#50926

@ntkme
Copy link
Member

ntkme commented Feb 2, 2023

sass-embedded now works out of box in nix-shell -p ruby:

(nixos/nix)root@6a397e63fe0a:/home/opc# nix-shell -p ruby
these 20 paths will be fetched (39.06 MiB download, 205.04 MiB unpacked):
  /nix/store/026xvlylahwkrcrsykh8fb5l08zfhqdr-binutils-wrapper-2.39
  /nix/store/0h6nfjybmhyx4d5vy9cyh0w9gmw67agd-expand-response-params
  /nix/store/19iw70icl1292rn7w20ghxbj3zrfkm8d-glibc-2.35-163-dev
  /nix/store/1g5g7vzlcprmg1d6864fs5r4nf1h5b5j-binutils-2.39
  /nix/store/4x5fcbxcxx8lgq3j635kwdfi8lpw6cnx-gnumake-4.3
  /nix/store/66rxqbbmk95pwcqmi9qqdcvlr6g1yalg-file-5.43
  /nix/store/7a84lbpavhwwrwf60nv0wvl2ffn5hg60-gcc-9.5.0
  /nix/store/894c2wg3zp2i0l1jfl09pdzxkq8pydjm-patchelf-0.15.0
  /nix/store/8h5wmlxyi78zc49g4k81gwsalrghk254-diffutils-3.8
  /nix/store/8v6p0if20h1xkvh8cdxb1fdz5g8iv10p-hook
  /nix/store/jh397z3hkn7gwyc1vaygfw8z9rx72ppk-stdenv-linux
  /nix/store/m1bi5njjbcchycnpvvil4vczmcbm6wcr-gawk-5.1.1
  /nix/store/na2lfq311l25ywvvcmacfiw8iccci8fb-ruby-2.7.6
  /nix/store/r86hfms9h6q8hzzrms0j89hncv3jxwff-gcc-wrapper-9.5.0
  /nix/store/vf3ry867iy52h3lm9nj43xnvfkfhbnyh-ed-1.18
  /nix/store/xf4ahkzxis0vvxnf98l3mal300jvrf3b-libyaml-0.2.5
  /nix/store/yi0ma6c3d8vzkv22kir44fnwqp1mkx1r-gnu-config-2021-01-25
  /nix/store/zbbx6ysrvnac4v3dq4gw211d6cfi03v4-patch-2.7.6
copying path '/nix/store/66rxqbbmk95pwcqmi9qqdcvlr6g1yalg-file-5.43' from 'https://cache.nixos.org'...
copying path '/nix/store/1g5g7vzlcprmg1d6864fs5r4nf1h5b5j-binutils-2.39' from 'https://cache.nixos.org'...
copying path '/nix/store/8h5wmlxyi78zc49g4k81gwsalrghk254-diffutils-3.8' from 'https://cache.nixos.org'...
copying path '/nix/store/vf3ry867iy52h3lm9nj43xnvfkfhbnyh-ed-1.18' from 'https://cache.nixos.org'...
copying path '/nix/store/0h6nfjybmhyx4d5vy9cyh0w9gmw67agd-expand-response-params' from 'https://cache.nixos.org'...
copying path '/nix/store/m1bi5njjbcchycnpvvil4vczmcbm6wcr-gawk-5.1.1' from 'https://cache.nixos.org'...
copying path '/nix/store/yi0ma6c3d8vzkv22kir44fnwqp1mkx1r-gnu-config-2021-01-25' from 'https://cache.nixos.org'...
copying path '/nix/store/4x5fcbxcxx8lgq3j635kwdfi8lpw6cnx-gnumake-4.3' from 'https://cache.nixos.org'...
copying path '/nix/store/8v6p0if20h1xkvh8cdxb1fdz5g8iv10p-hook' from 'https://cache.nixos.org'...
copying path '/nix/store/xf4ahkzxis0vvxnf98l3mal300jvrf3b-libyaml-0.2.5' from 'https://cache.nixos.org'...
copying path '/nix/store/9rimms4jxabcizxlvplniwik1a3q683r-linux-headers-6.0' from 'https://cache.nixos.org'...
copying path '/nix/store/jjgqbq81kn8lh3h4pm7mdnn816a6182q-openssl-1.1.1s' from 'https://cache.nixos.org'...
copying path '/nix/store/19iw70icl1292rn7w20ghxbj3zrfkm8d-glibc-2.35-163-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/zbbx6ysrvnac4v3dq4gw211d6cfi03v4-patch-2.7.6' from 'https://cache.nixos.org'...
copying path '/nix/store/026xvlylahwkrcrsykh8fb5l08zfhqdr-binutils-wrapper-2.39' from 'https://cache.nixos.org'...
copying path '/nix/store/7a84lbpavhwwrwf60nv0wvl2ffn5hg60-gcc-9.5.0' from 'https://cache.nixos.org'...
copying path '/nix/store/894c2wg3zp2i0l1jfl09pdzxkq8pydjm-patchelf-0.15.0' from 'https://cache.nixos.org'...
copying path '/nix/store/r86hfms9h6q8hzzrms0j89hncv3jxwff-gcc-wrapper-9.5.0' from 'https://cache.nixos.org'...
copying path '/nix/store/na2lfq311l25ywvvcmacfiw8iccci8fb-ruby-2.7.6' from 'https://cache.nixos.org'...
copying path '/nix/store/jh397z3hkn7gwyc1vaygfw8z9rx72ppk-stdenv-linux' from 'https://cache.nixos.org'...

[nix-shell:/home/opc]# gem install sass-embedded
Fetching sass-embedded-1.58.0.gem
Fetching google-protobuf-3.21.12.gem
Building native extensions. This could take a while...
Successfully installed google-protobuf-3.21.12
Building native extensions. This could take a while...
Successfully installed sass-embedded-1.58.0

Parsing documentation for google-protobuf-3.21.12
unable to convert "\xDB" from ASCII-8BIT to UTF-8 for lib/google/protobuf/descriptor_pb.rb, skipping
Installing ri documentation for google-protobuf-3.21.12
Parsing documentation for sass-embedded-1.58.0
Installing ri documentation for sass-embedded-1.58.0
Done installing documentation for google-protobuf, sass-embedded after 0 seconds
2 gems installed

[nix-shell:/home/opc]# irb
irb(main):001:0> require 'sass-embedded'
=> true
irb(main):002:0> Sass.info
=> "sass-embedded\t1.58.0"
irb(main):003:0> 

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants