Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hello WebAssembly! (MVP implementation) #10870

Merged
merged 48 commits into from
Mar 21, 2022
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
cabdb42
Add Wasm32 LibC (based on musl)
lbguilherme Jul 1, 2021
76d5e22
Add Basic Wasm32 support
lbguilherme Jul 1, 2021
98c5447
Add LibC as a symbolic link
lbguilherme Jul 1, 2021
94109db
Merge branch 'master' into feat/webassembly
lbguilherme Jul 14, 2021
fa5077a
Move wasm to a Crystal::System implementation
lbguilherme Jul 14, 2021
fe511de
Enable smoke tests for wasm32-wasi
lbguilherme Jul 14, 2021
165b974
Revert change on unix Thread
lbguilherme Jul 14, 2021
be594ed
Smoke test: build compiler for the wasm test
lbguilherme Jul 14, 2021
ffe4b5b
Merge remote-tracking branch 'upstream/master' into feat/webassembly
lbguilherme Jul 20, 2021
598c99f
Mark stub methods with NotImplementedError or TODO
lbguilherme Jul 21, 2021
2d78c9b
Update src/crystal/system/wasm/process.cr
lbguilherme Jul 21, 2021
ffd9c12
Update wasi libc: it's now limited to what wasi actually provides. Mo…
lbguilherme Jul 22, 2021
9faf715
Use arc4random instead of getentropy
lbguilherme Jul 22, 2021
b0797c1
Remove Signal, Process and Sockets unimplemented method from Wasm32
lbguilherme Aug 2, 2021
aec74ad
Work around compiler bug in previous commit: BUG: `typeof(_io)` at ex…
lbguilherme Aug 2, 2021
463d4cd
fix: cleanup from previous commit
lbguilherme Aug 2, 2021
5faf601
Merge remote-tracking branch 'upstream/master' into feat/webassembly
lbguilherme Aug 2, 2021
3d91387
Merge remote-tracking branch 'upstream/master' into feat/webassembly
lbguilherme Aug 3, 2021
b1dd5f7
crystal tool format
lbguilherme Aug 3, 2021
00b10d6
fix: use correct .wasm extension
lbguilherme Dec 18, 2021
eab4739
Merge remote-tracking branch 'upstream/master' into feat/webassembly
lbguilherme Dec 18, 2021
4e33d92
Merge remote-tracking branch 'upstream/master' into feat/webassembly
lbguilherme Dec 23, 2021
e86b675
feat: implement Crystal::System::Fiber with malloc/free
lbguilherme Dec 23, 2021
ba5a531
fix: mark system_linger as not implemented
lbguilherme Dec 24, 2021
2582817
feat: add direct wasi integration for Dir.entries
lbguilherme Dec 24, 2021
4d5b061
feat: enable libpcre
lbguilherme Dec 24, 2021
5b4627f
chore: refactor Crystal::Program.object_extension
lbguilherme Dec 28, 2021
2519a6a
fix: mock Thread.current
lbguilherme Dec 28, 2021
fdef3ac
chore: change PREOPENS to @@preopens
lbguilherme Dec 28, 2021
10c6eb7
fix: print fatal errors
lbguilherme Dec 28, 2021
a0407ca
fix: use Sync logger until channels work
lbguilherme Dec 28, 2021
c3f035f
chore: rename wasierror.cr -> wasi_error.cr
lbguilherme Dec 28, 2021
3ba1b96
chore: preopen cleanup
lbguilherme Dec 28, 2021
e58cf26
fix: there is no need to initialize PREOPENS as early as possible if …
lbguilherme Dec 29, 2021
7bcd75a
fix: missing wasi require
lbguilherme Dec 29, 2021
cb2ea23
feat: generate better linker command
lbguilherme Dec 29, 2021
ad2e402
chore: stub Exception::CallStack#decode_backtrace until callback support
lbguilherme Dec 29, 2021
2e4b517
chore: remove some unused pieces from wasm libc
lbguilherme Dec 31, 2021
5faa13a
fix: use "out fd"
lbguilherme Jan 27, 2022
9de1150
feat: add wasi random_get to avoid relying on arc4random polyfill
lbguilherme Feb 8, 2022
2d3158e
Mark WASI as a UNIX target and use it as the system environment, inst…
lbguilherme Feb 8, 2022
ac74bad
Update spec/compiler/loader/unix_spec.cr
lbguilherme Feb 8, 2022
9f72ef9
Update spec/compiler/ffi/ffi_spec.cr
lbguilherme Feb 8, 2022
fea289a
Revert "Update spec/compiler/loader/unix_spec.cr"
lbguilherme Feb 8, 2022
53f6c25
Revert "Update spec/compiler/ffi/ffi_spec.cr"
lbguilherme Feb 8, 2022
5995375
fix: correct usage of some wasm32<->wasi flags
lbguilherme Feb 8, 2022
570eb32
Merge remote-tracking branch 'upstream/master' into feat/webassembly
lbguilherme Feb 11, 2022
b011761
fix: home_path for wasi
lbguilherme Feb 11, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion spec/compiler/ffi/ffi_spec.cr
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{% skip_file if !flag?(:unix) || flag?(:without_ffi) %}
{% skip_file if !flag?(:unix) || flag?(:without_ffi) || flag?(:wasm32) %}
lbguilherme marked this conversation as resolved.
Show resolved Hide resolved

require "../spec_helper"
require "compiler/crystal/ffi"
Expand Down
2 changes: 1 addition & 1 deletion spec/compiler/loader/unix_spec.cr
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{% skip_file unless flag?(:unix) %}
{% skip_file if !flag?(:unix) || flag?(:wasm32) %}
lbguilherme marked this conversation as resolved.
Show resolved Hide resolved

require "./spec_helper"
require "../spec_helper"
Expand Down
6 changes: 5 additions & 1 deletion src/compiler/crystal/codegen/target.cr
Original file line number Diff line number Diff line change
Expand Up @@ -96,12 +96,16 @@ class Crystal::Codegen::Target
@environment.starts_with?("linux")
end

def wasi?
@environment.starts_with?("wasi")
end

def bsd?
freebsd? || netbsd? || openbsd? || dragonfly?
end

def unix?
macos? || bsd? || linux?
macos? || bsd? || linux? || wasi?
end

def gnu?
Expand Down
8 changes: 4 additions & 4 deletions src/crystal/system.cr
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,10 @@ module Crystal::System
# def self.cpu_count
end

{% if flag?(:unix) %}
{% if flag?(:wasi) %}
require "./system/wasi/hostname"
require "./system/wasi/cpucount"
{% elsif flag?(:unix) %}
require "./system/unix/hostname"

{% if flag?(:bsd) %}
Expand All @@ -19,9 +22,6 @@ end
{% elsif flag?(:win32) %}
require "./system/win32/hostname"
require "./system/win32/cpucount"
{% elsif flag?(:wasm32) %}
require "./system/wasm/hostname"
require "./system/wasm/cpucount"
{% else %}
{% raise "No Crystal::System implementation available" %}
{% end %}
6 changes: 3 additions & 3 deletions src/crystal/system/dir.cr
Original file line number Diff line number Diff line change
Expand Up @@ -51,10 +51,10 @@ module Crystal::System::Dir
# def self.delete(path : String) : Nil
end

{% if flag?(:unix) %}
{% if flag?(:wasi) %}
require "./wasi/dir"
{% elsif flag?(:unix) %}
require "./unix/dir"
{% elsif flag?(:wasm32) %}
require "./wasm/dir"
{% elsif flag?(:win32) %}
require "./win32/dir"
{% else %}
Expand Down
2 changes: 1 addition & 1 deletion src/crystal/system/env.cr
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ module Crystal::System::Env
# def self.each(&block : String, String ->)
end

{% if flag?(:unix) || flag?(:wasm32) %}
{% if flag?(:unix) %}
require "./unix/env"
{% elsif flag?(:win32) %}
require "./win32/env"
Expand Down
4 changes: 2 additions & 2 deletions src/crystal/system/event_loop.cr
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ struct Crystal::Event
# def add(time_span : Time::Span?) : Nil
end

{% if flag?(:wasm32) %}
require "./wasm/event_loop"
{% if flag?(:wasi) %}
require "./wasi/event_loop"
{% elsif flag?(:unix) %}
require "./unix/event_loop_libevent"
{% elsif flag?(:win32) %}
Expand Down
6 changes: 3 additions & 3 deletions src/crystal/system/fiber.cr
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@ module Crystal::System::Fiber
# def self.main_fiber_stack(stack_bottom : Void*) : Void*
end

{% if flag?(:unix) %}
{% if flag?(:wasi) %}
require "./wasi/fiber"
{% elsif flag?(:unix) %}
require "./unix/fiber"
{% elsif flag?(:win32) %}
require "./win32/fiber"
{% elsif flag?(:wasm32) %}
require "./wasm/fiber"
{% else %}
{% raise "fiber not supported" %}
{% end %}
6 changes: 3 additions & 3 deletions src/crystal/system/file.cr
Original file line number Diff line number Diff line change
Expand Up @@ -49,12 +49,12 @@ module Crystal::System::File
# def file_descriptor_close
end

{% if flag?(:unix) %}
{% if flag?(:wasi) %}
require "./wasi/file"
{% elsif flag?(:unix) %}
require "./unix/file"
{% elsif flag?(:win32) %}
require "./win32/file"
{% elsif flag?(:wasm32) %}
require "./wasm/file"
{% else %}
{% raise "No Crystal::System::File implementation available" %}
{% end %}
6 changes: 3 additions & 3 deletions src/crystal/system/file_descriptor.cr
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
{% if flag?(:unix) %}
{% if flag?(:wasi) %}
require "./wasi/file_descriptor"
{% elsif flag?(:unix) %}
require "./unix/file_descriptor"
{% elsif flag?(:win32) %}
require "./win32/file_descriptor"
{% elsif flag?(:wasm32) %}
require "./wasm/file_descriptor"
{% else %}
{% raise "No Crystal::System::FileDescriptor implementation available" %}
{% end %}
2 changes: 1 addition & 1 deletion src/crystal/system/file_info.cr
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{% if flag?(:unix) || flag?(:wasm32) %}
{% if flag?(:unix) %}
require "./unix/file_info"
{% elsif flag?(:win32) %}
require "./win32/file_info"
Expand Down
6 changes: 3 additions & 3 deletions src/crystal/system/group.cr
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{% if flag?(:unix) %}
{% if flag?(:wasi) %}
require "./wasi/group"
{% elsif flag?(:unix) %}
require "./unix/group"
{% elsif flag?(:wasm32) %}
require "./wasm/group"
{% else %}
{% raise "No Crystal::System::Group implementation available" %}
{% end %}
2 changes: 1 addition & 1 deletion src/crystal/system/mime.cr
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ module Crystal::System::MIME
# def self.load
end

{% if flag?(:unix) || flag?(:wasm32) %}
{% if flag?(:unix) %}
require "./unix/mime"
{% elsif flag?(:win32) %}
require "./win32/mime"
Expand Down
2 changes: 1 addition & 1 deletion src/crystal/system/path.cr
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ module Crystal::System::Path
# def self.home : String
end

{% if flag?(:unix) || flag?(:wasm32) %}
{% if flag?(:unix) %}
require "./unix/path"
{% elsif flag?(:win32) %}
require "./win32/path"
Expand Down
2 changes: 1 addition & 1 deletion src/crystal/system/print_error.cr
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ module Crystal::System
# This is useful for error messages from components that are required for
# IO to work (fibers, scheduler, event_loop).
def self.print_error(message, *args)
{% if flag?(:unix) || flag?(:wasm32) %}
{% if flag?(:unix) %}
LibC.dprintf 2, message, *args
{% elsif flag?(:win32) %}
buffer = StaticArray(UInt8, 512).new(0_u8)
Expand Down
6 changes: 3 additions & 3 deletions src/crystal/system/process.cr
Original file line number Diff line number Diff line change
Expand Up @@ -70,12 +70,12 @@ module Crystal::System
ORIGINAL_STDERR = IO::FileDescriptor.new(2, blocking: true)
end

{% if flag?(:unix) %}
{% if flag?(:wasi) %}
require "./wasi/process"
{% elsif flag?(:unix) %}
require "./unix/process"
{% elsif flag?(:win32) %}
require "./win32/process"
{% elsif flag?(:wasm32) %}
require "./wasm/process"
{% else %}
{% raise "No Crystal::System::Process implementation available" %}
{% end %}
6 changes: 4 additions & 2 deletions src/crystal/system/random.cr
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,11 @@ module Crystal::System::Random
# def self.next_u
end

{% if flag?(:linux) %}
{% if flag?(:wasi) %}
require "./wasi/random"
{% elsif flag?(:linux) %}
require "./unix/getrandom"
{% elsif flag?(:openbsd) || flag?(:netbsd) || flag?(:wasm32) %}
{% elsif flag?(:openbsd) || flag?(:netbsd) %}
require "./unix/arc4random"
{% elsif flag?(:unix) %}
require "./unix/urandom"
Expand Down
6 changes: 3 additions & 3 deletions src/crystal/system/socket.cr
Original file line number Diff line number Diff line change
Expand Up @@ -76,10 +76,10 @@ module Crystal::System::Socket
# private def system_tcp_keepalive_count=(val : Int)
end

{% if flag?(:unix) %}
{% if flag?(:wasi) %}
require "./wasi/socket"
{% elsif flag?(:unix) %}
require "./unix/socket"
{% elsif flag?(:wasm32) %}
require "./wasm/socket"
{% elsif flag?(:win32) %}
require "./win32/socket"
{% else %}
Expand Down
6 changes: 3 additions & 3 deletions src/crystal/system/thread.cr
Original file line number Diff line number Diff line change
Expand Up @@ -28,13 +28,13 @@ end

require "./thread_linked_list"

{% if flag?(:unix) %}
{% if flag?(:wasi) %}
require "./wasi/thread"
{% elsif flag?(:unix) %}
require "./unix/pthread"
require "./unix/pthread_condition_variable"
{% elsif flag?(:win32) %}
require "./win32/thread"
{% elsif flag?(:wasm32) %}
require "./wasm/thread"
{% else %}
{% raise "thread not supported" %}
{% end %}
6 changes: 3 additions & 3 deletions src/crystal/system/thread_mutex.cr
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,12 @@ class Thread
end
end

{% if flag?(:unix) %}
{% if flag?(:wasi) %}
require "./wasi/thread_mutex"
{% elsif flag?(:unix) %}
require "./unix/pthread_mutex"
{% elsif flag?(:win32) %}
require "./win32/thread_mutex"
{% elsif flag?(:wasm32) %}
require "./wasm/thread_mutex"
{% else %}
{% raise "thread not supported" %}
{% end %}
2 changes: 1 addition & 1 deletion src/crystal/system/time.cr
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ module Crystal::System::Time
# def self.load_localtime : ::Time::Location?
end

{% if flag?(:unix) || flag?(:wasm32) %}
{% if flag?(:unix) %}
require "./unix/time"
{% elsif flag?(:win32) %}
require "./win32/time"
Expand Down
6 changes: 3 additions & 3 deletions src/crystal/system/user.cr
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{% if flag?(:unix) %}
{% if flag?(:wasi) %}
require "./wasi/user"
{% elsif flag?(:unix) %}
require "./unix/user"
{% elsif flag?(:wasm32) %}
require "./wasm/user"
{% else %}
{% raise "No Crystal::System::User implementation available" %}
{% end %}
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -98,4 +98,5 @@ lib LibWasi
fun path_open = __wasi_path_open(fd : Fd, dirflags : LookupFlags, path : UInt8*, oflags : OpenFlags, fs_rights_base : Rights, fs_rights_inheriting : Rights, fdflags : FdFlags, ret : Fd*) : WasiError
fun fd_readdir = __wasi_fd_readdir(fd : Fd, buf : UInt8*, len : Size, cookie : UInt64, ret : Size*) : WasiError
fun fd_close = __wasi_fd_close(fd : Fd) : WasiError
fun random_get = __wasi_random_get(buf : UInt8*, len : Size) : WasiError
end
File renamed without changes.
14 changes: 14 additions & 0 deletions src/crystal/system/wasi/random.cr
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
require "./lib_wasi"

module Crystal::System::Random
def self.random_bytes(buf : Bytes) : Nil
err = LibWasi.random_get(buf, buf.size)
raise RuntimeError.from_os_error("random_get", err) unless err.success?
end

def self.next_u : UInt8
buf = uninitialized UInt8[1]
random_bytes(buf.to_slice)
buf.unsafe_as(UInt8)
end
end
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,30 +1,13 @@
class Thread
# all thread objects, so the GC can see them (it doesn't scan thread locals)
@@threads = Thread::LinkedList(Thread).new

@exception : Exception?
@detached = Atomic(UInt8).new(0)
@main_fiber : Fiber?
@func : (Proc(Nil))?

# :nodoc:
property next : Thread?

# :nodoc:
property previous : Thread?

def self.unsafe_each
@@threads.unsafe_each { |thread| yield thread }
end

def initialize
@main_fiber = Fiber.new(stack_address, self)
@@threads.push(self)

# TODO: Create thread
end

def initialize(&@func : ->)
def initialize(&func : ->)
initialize
end

Expand All @@ -36,15 +19,15 @@ class Thread
raise NotImplementedError.new("Thread.yield")
end

@@current : Thread? = Thread.new
@@current = Thread.new

# Associates the Thread object to the running system thread.
protected def self.current=(@@current : Thread) : Thread
end

# Returns the Thread object associated to the running system thread.
def self.current : Thread
@@current || raise "BUG: Thread.current returned NULL"
@@current
end

# Create the thread object for the current thread (aka the main thread of the
Expand All @@ -64,18 +47,7 @@ class Thread
end

protected def start
Thread.current = self
@main_fiber = fiber = Fiber.new(stack_address, self)

begin
@func.call
rescue ex
@exception = ex
ensure
@@threads.delete(self)
Fiber.inactive(fiber)
detach_self
end
raise NotImplementedError.new("Thread#start")
end

private def stack_address : Void*
Expand Down
File renamed without changes.
File renamed without changes.