Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Emscripten stack simplification #1870

Merged
merged 5 commits into from
Jan 16, 2019
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion auto_update_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ def update_lld_tests():
if ext != '.out' and not os.path.exists(out_path):
continue
cmd = (WASM_EMSCRIPTEN_FINALIZE +
[wast_path, '-S', '--global-base=568'] + ext_args)
[wast_path, '-S', '--global-base=568', '--stack-base=16384'] + ext_args)
actual = run_command(cmd)
with open(out_path, 'w') as o:
o.write(actual)
Expand Down
2 changes: 1 addition & 1 deletion scripts/test/lld.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ def test_wasm_emscripten_finalize():
continue

cmd = (WASM_EMSCRIPTEN_FINALIZE +
[wast_path, '-S', '--global-base=568'] + ext_args)
[wast_path, '-S', '--global-base=568', '--stack-base=16384'] + ext_args)
actual = run_command(cmd)

if not os.path.exists(expected_file):
Expand Down
10 changes: 6 additions & 4 deletions src/asm2wasm.h
Original file line number Diff line number Diff line change
Expand Up @@ -426,13 +426,16 @@ class Asm2WasmBuilder {
std::map<IString, MappedGlobal> mappedGlobals;

private:
void allocateGlobal(IString name, Type type) {
void allocateGlobal(IString name, Type type, Literal value=Literal()) {
assert(mappedGlobals.find(name) == mappedGlobals.end());
if (value.type == none) {
value = Literal::makeZero(type);
}
mappedGlobals.emplace(name, MappedGlobal(type));
wasm.addGlobal(builder.makeGlobal(
name,
type,
LiteralUtils::makeZero(type, wasm),
builder.makeConst(value),
Builder::Mutable
));
}
Expand Down Expand Up @@ -986,8 +989,7 @@ void Asm2WasmBuilder::processAsm(Ref ast) {
Ref value = pair[1];
if (value->isNumber()) {
// global int
assert(value->getNumber() == 0);
allocateGlobal(name, Type::i32);
allocateGlobal(name, Type::i32, Literal(int32_t(value->getInteger())));
} else if (value[0] == BINARY) {
// int import
assert(value[1] == OR && value[3]->isNumber() && value[3]->getNumber() == 0);
Expand Down
20 changes: 17 additions & 3 deletions src/tools/wasm-emscripten-finalize.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ using namespace cashew;
using namespace wasm;

int main(int argc, const char *argv[]) {
const uint64_t INVALID_BASE = -1;

std::string infile;
std::string outfile;
std::string inputSourceMapFilename;
Expand All @@ -46,7 +48,8 @@ int main(int argc, const char *argv[]) {
bool debugInfo = false;
bool legalizeJavaScriptFFI = true;
unsigned numReservedFunctionPointers = 0;
uint64_t globalBase;
uint64_t globalBase = INVALID_BASE;
uint64_t stackBase = INVALID_BASE;
Options options("wasm-emscripten-finalize",
"Performs Emscripten-specific transforms on .wasm files");
options
Expand Down Expand Up @@ -75,11 +78,16 @@ int main(int argc, const char *argv[]) {
const std::string &argument) {
numReservedFunctionPointers = std::stoi(argument);
})
.add("--global-base", "", "Where lld started to place globals",
.add("--global-base", "", "Where lld should start to place static globals",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This command seems wrong. lld has already assigned all its globals before finalize is called.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see we pass the global base to both wasm-emscripten-finalize and to lld - is lld doing it in both bitcode linking and wasm object files? If so I can remove the flag to finalize.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only reason I can see that finalize needs to know this is because it wants to write "staticBump" to the metadata output about the binary. as long as we need staticBump we will to continue to pass this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i see, thanks. Ok, fixing the comment.

Options::Arguments::One,
[&globalBase](Options*, const std::string&argument ) {
globalBase = std::stoull(argument);
})
.add("--stack-base", "", "Where the stack begins",
Options::Arguments::One,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this new argument? Won't the existing stack pointer assigned by lld be correct?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We tell lld the size of the stack, but not its absolute location. Instead, finalize adds a wasm import of STACKTOP, and JS sends it over, and then the import gets assigned to the stack pointer lld created (which is mutable, so that way we couldn't assign it directly). This PR changes that, to just hardcode an initial value for the stack pointer, avoiding the import.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, are you saying, lld guesses the location of the stack (right after static allocations), and we can just leave it?

Even so, that wouldn't be right, as emscripten can add more static allocations after lld (in the JS compiler). Even without that, it seems safest to have one location that decides this stuff (Memory() in emscripten.py) and informs everything else.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But in this new world, isn't the stack pointer already set by lld to the correct value? Does emscripten put stuff on the stack at startup? If so, doesn't it already have a way of moving the stack pointer by calling stackSave/stackRestore?

Perhaps this argument should be called "--initial-stack-pointer"?

I'm happy with the PR going in either way as I think its a step forward, I'm just trying to see possible improvements going foreword.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lld's guess at the stack pointer location is incorrect due to static allocations from JS (see my last comment, may have been a race).

Emscripten won't use the stack from JS, as the wasm controls it - it needs to call into wasm to get the stack pointer and modify it. So it can modify it after the wasm is ready and the program started up.

Good idea to rename this to --initial-stack-pointer, done.

Copy link
Member

@sbc100 sbc100 Jan 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that the memory layout used by lld/wasm-backend is

1.  zero page
2.  static initialized data
3.  bss data
4.  stack (grows down)
5.  heap 

I believe that emscripten adds global data after (3), which mean its clobber the end of the stack (since the stack growns down). So I believe the inital stack pointer set by lld (which grows down) will always be correct... but I'm not totally sure about all this right now.

Happy for this change to land as is and we can try to simplify and possible remove this argument.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, emscripten adds more stuff after 3. But it keeps the stack size fixed while doing so, so the initial stack location becomes higher - it pushes up the entire range reserved for the stack, top and bottom.

[&stackBase](Options*, const std::string&argument ) {
stackBase = std::stoull(argument);
})

.add("--input-source-map", "-ism", "Consume source map from the specified file",
Options::Arguments::One,
Expand Down Expand Up @@ -141,6 +149,12 @@ int main(int argc, const char *argv[]) {
uint32_t dataSize = 0;

if (!isSideModule) {
if (globalBase == INVALID_BASE) {
Fatal() << "globalBase must be set";
}
if (stackBase == INVALID_BASE) {
Fatal() << "stackBase must be set";
}
Export* dataEndExport = wasm.getExport("__data_end");
if (dataEndExport == nullptr) {
Fatal() << "__data_end export not found";
Expand Down Expand Up @@ -189,7 +203,7 @@ int main(int argc, const char *argv[]) {
} else {
generator.generateRuntimeFunctions();
generator.generateMemoryGrowthFunction();
generator.generateStackInitialization();
generator.generateStackInitialization(stackBase);
// emscripten calls this by default for side libraries so we only need
// to include in as a static ctor for main module case.
if (wasm.getExportOrNull("__post_instantiate")) {
Expand Down
2 changes: 1 addition & 1 deletion src/wasm-emscripten.h
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ class EmscriptenGlueGenerator {

void generateRuntimeFunctions();
Function* generateMemoryGrowthFunction();
void generateStackInitialization();
void generateStackInitialization(Address addr);

// Create thunks for use with emscripten Runtime.dynCall. Creates one for each
// signature in the indirect function table.
Expand Down
14 changes: 2 additions & 12 deletions src/wasm/wasm-emscripten.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -156,19 +156,9 @@ Function* EmscriptenGlueGenerator::generateMemoryGrowthFunction() {
return growFunction;
}

void EmscriptenGlueGenerator::generateStackInitialization() {
// Replace a global with a constant initial value with an imported
// initial value, which emscripten JS will send us.
// TODO: with mutable imported globals, we can avoid adding another
// global for the import.
Builder builder(wasm);
auto* import = builder.makeGlobal(STACK_INIT, i32, nullptr, Builder::Immutable);
import->module = ENV;
import->base = STACKTOP;
wasm.addGlobal(import);
void EmscriptenGlueGenerator::generateStackInitialization(Address addr) {
auto* stackPointer = getStackPointerGlobal();
assert(stackPointer->init->is<Const>());
stackPointer->init = builder.makeGetGlobal(import->name, i32);
stackPointer->init->cast<Const>()->value = Literal(int32_t(addr));
}

static bool hasI64ResultOrParam(FunctionType* ft) {
Expand Down
3 changes: 1 addition & 2 deletions test/lld/duplicate_imports.wast.out
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,14 @@
(type $legaltype$puts2 (func (param i32 i32) (result i32)))
(type $legaltype$invoke_ffd (func (param i32 f64 f64) (result f64)))
(type $legaltype$invoke_ffd2 (func (param i32 f64 f64) (result f64)))
(import "env" "STACKTOP" (global $stack$init i32))
(import "env" "puts" (func $puts1 (param i32) (result i32)))
(import "env" "puts" (func $legalimport$puts2 (param i32 i32) (result i32)))
(import "env" "invoke_ffd" (func $legalimport$invoke_ffd (param i32 f64 f64) (result f64)))
(import "env" "invoke_ffd" (func $legalimport$invoke_ffd2 (param i32 f64 f64) (result f64)))
(memory $0 2)
(data (i32.const 568) "Hello, world\00")
(table $0 1 1 funcref)
(global $global$0 (mut i32) (global.get $stack$init))
(global $global$0 (mut i32) (i32.const 16384))
(global $global$1 i32 (i32.const 66128))
(global $global$2 i32 (i32.const 581))
(export "memory" (memory $0))
Expand Down
3 changes: 1 addition & 2 deletions test/lld/em_asm.wast.out
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,13 @@
(type $FUNCSIG$ii (func (param i32) (result i32)))
(type $FUNCSIG$iiii (func (param i32 i32 i32) (result i32)))
(type $FUNCSIG$iii (func (param i32 i32) (result i32)))
(import "env" "STACKTOP" (global $stack$init i32))
(import "env" "emscripten_asm_const_i" (func $emscripten_asm_const_i (param i32) (result i32)))
(import "env" "emscripten_asm_const_iii" (func $emscripten_asm_const_iii (param i32 i32 i32) (result i32)))
(import "env" "emscripten_asm_const_ii" (func $emscripten_asm_const_ii (param i32 i32) (result i32)))
(memory $0 2)
(data (i32.const 568) "{ Module.print(\"Hello world\"); }\00{ return $0 + $1; }\00{ Module.print(\"Got \" + $0); }\00")
(table $0 1 1 funcref)
(global $global$0 (mut i32) (global.get $stack$init))
(global $global$0 (mut i32) (i32.const 16384))
(global $global$1 i32 (i32.const 66192))
(global $global$2 i32 (i32.const 652))
(export "memory" (memory $0))
Expand Down
3 changes: 1 addition & 2 deletions test/lld/em_asm_table.wast.out
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,11 @@
(type $FUNCSIG$vii (func (param i32 i32)))
(type $FUNCSIG$iiii (func (param i32 i32 i32) (result i32)))
(import "env" "memory" (memory $0 8192))
(import "env" "STACKTOP" (global $stack$init i32))
(import "env" "emscripten_log" (func $fimport$0 (param i32 i32)))
(import "env" "emscripten_asm_const_iii" (func $emscripten_asm_const_iii (param i32 i32 i32) (result i32)))
(table $0 159609 funcref)
(elem (i32.const 1) $fimport$0 $emscripten_asm_const_iii)
(global $global$0 (mut i32) (global.get $stack$init))
(global $global$0 (mut i32) (i32.const 16384))
(global $global$1 i32 (i32.const 1048))
(export "__data_end" (global $global$1))
(export "stackSave" (func $stackSave))
Expand Down
3 changes: 1 addition & 2 deletions test/lld/hello_world.wast.mem.out
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,10 @@
(type $1 (func (result i32)))
(type $2 (func))
(type $FUNCSIG$ii (func (param i32) (result i32)))
(import "env" "STACKTOP" (global $stack$init i32))
(import "env" "puts" (func $puts (param i32) (result i32)))
(memory $0 2)
(table $0 1 1 funcref)
(global $global$0 (mut i32) (global.get $stack$init))
(global $global$0 (mut i32) (i32.const 16384))
(global $global$1 i32 (i32.const 66128))
(global $global$2 i32 (i32.const 581))
(export "memory" (memory $0))
Expand Down
3 changes: 1 addition & 2 deletions test/lld/hello_world.wast.out
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,11 @@
(type $1 (func (result i32)))
(type $2 (func))
(type $FUNCSIG$ii (func (param i32) (result i32)))
(import "env" "STACKTOP" (global $stack$init i32))
(import "env" "puts" (func $puts (param i32) (result i32)))
(memory $0 2)
(data (i32.const 568) "Hello, world\00")
(table $0 1 1 funcref)
(global $global$0 (mut i32) (global.get $stack$init))
(global $global$0 (mut i32) (i32.const 16384))
(global $global$1 i32 (i32.const 66128))
(global $global$2 i32 (i32.const 581))
(export "memory" (memory $0))
Expand Down
3 changes: 1 addition & 2 deletions test/lld/init.wast.out
Original file line number Diff line number Diff line change
@@ -1,11 +1,10 @@
(module
(type $0 (func))
(type $1 (func (result i32)))
(import "env" "STACKTOP" (global $stack$init i32))
(memory $0 2)
(data (i32.const 568) "\00\00\00\00\00\00\00\00")
(table $0 1 1 funcref)
(global $global$0 (mut i32) (global.get $stack$init))
(global $global$0 (mut i32) (i32.const 16384))
(global $global$1 i32 (i32.const 66112))
(global $global$2 i32 (i32.const 576))
(export "memory" (memory $0))
Expand Down
3 changes: 1 addition & 2 deletions test/lld/recursive.wast.out
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,11 @@
(type $1 (func (result i32)))
(type $2 (func))
(type $FUNCSIG$iii (func (param i32 i32) (result i32)))
(import "env" "STACKTOP" (global $stack$init i32))
(import "env" "printf" (func $printf (param i32 i32) (result i32)))
(memory $0 2)
(data (i32.const 568) "%d:%d\n\00Result: %d\n\00")
(table $0 1 1 funcref)
(global $global$0 (mut i32) (global.get $stack$init))
(global $global$0 (mut i32) (i32.const 16384))
(global $global$1 i32 (i32.const 66128))
(global $global$2 i32 (i32.const 587))
(export "memory" (memory $0))
Expand Down
3 changes: 1 addition & 2 deletions test/lld/reserved_func_ptr.wast.jscall.out
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@
(type $FUNCSIG$v (func))
(type $FUNCSIG$vii (func (param i32 i32)))
(type $FUNCSIG$viiii (func (param i32 i32 i32 i32)))
(import "env" "STACKTOP" (global $stack$init i32))
(import "env" "_Z4atoiPKc" (func $_Z4atoiPKc (param i32) (result i32)))
(import "env" "jsCall_ddi" (func $jsCall_ddi (param i32 f64 i32) (result f64)))
(import "env" "jsCall_fffi" (func $jsCall_fffi (param i32 f32 f32 i32) (result f32)))
Expand All @@ -29,7 +28,7 @@
(memory $0 2)
(table $0 21 21 funcref)
(elem (i32.const 1) $_Z18address_taken_funciii $_Z19address_taken_func2iii $jsCall_ddi_0 $jsCall_ddi_1 $jsCall_ddi_2 $jsCall_fffi_0 $jsCall_fffi_1 $jsCall_fffi_2 $jsCall_iii_0 $jsCall_iii_1 $jsCall_iii_2 $jsCall_v_0 $jsCall_v_1 $jsCall_v_2 $jsCall_vi_0 $jsCall_vi_1 $jsCall_vi_2 $jsCall_viii_0 $jsCall_viii_1 $jsCall_viii_2)
(global $global$0 (mut i32) (global.get $stack$init))
(global $global$0 (mut i32) (i32.const 16384))
(global $global$1 i32 (i32.const 66112))
(global $global$2 i32 (i32.const 568))
(export "memory" (memory $0))
Expand Down
3 changes: 1 addition & 2 deletions test/lld/reserved_func_ptr.wast.out
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,11 @@
(type $6 (func (param i32) (result i32)))
(type $FUNCSIG$ii (func (param i32) (result i32)))
(type $FUNCSIG$viii (func (param i32 i32 i32)))
(import "env" "STACKTOP" (global $stack$init i32))
(import "env" "_Z4atoiPKc" (func $_Z4atoiPKc (param i32) (result i32)))
(memory $0 2)
(table $0 3 3 funcref)
(elem (i32.const 1) $_Z18address_taken_funciii $_Z19address_taken_func2iii)
(global $global$0 (mut i32) (global.get $stack$init))
(global $global$0 (mut i32) (i32.const 16384))
(global $global$1 i32 (i32.const 66112))
(global $global$2 i32 (i32.const 568))
(export "memory" (memory $0))
Expand Down
3 changes: 3 additions & 0 deletions test/unit.asm.js
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ function asm(global, env, buffer) {
var HEAPF32 = new global.Float32Array(buffer);
var HEAPF64 = new global.Float64Array(buffer);

var nonZero = 1337;

function big_negative() {
var temp = 0.0;
temp = +-2147483648;
Expand Down Expand Up @@ -785,6 +787,7 @@ function asm(global, env, buffer) {
emterpretify_assertions_safeHeap();
call_emscripten_log();
mod_detectSign(1.0, 2.31, 9.78);
nonZero = nonZero + 1 | 0;
}

function v() {
Expand Down
7 changes: 7 additions & 0 deletions test/unit.fromasm
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
(global $Int (mut i32) (i32.const 0))
(global $Double (mut f64) (f64.const 0))
(global $n (mut i32) (global.get $n$asm2wasm$import))
(global $nonZero (mut i32) (i32.const 1337))
(global $exportedNumber i32 (i32.const 42))
(export "big_negative" (func $big_negative))
(export "pick" (func $big_negative))
Expand Down Expand Up @@ -1162,6 +1163,12 @@
)
)
)
(global.set $nonZero
(i32.add
(global.get $nonZero)
(i32.const 1)
)
)
)
(func $vi (; 55 ;) (; has Stack IR ;) (param $0 i32)
(nop)
Expand Down
7 changes: 7 additions & 0 deletions test/unit.fromasm.clamp
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
(global $Int (mut i32) (i32.const 0))
(global $Double (mut f64) (f64.const 0))
(global $n (mut i32) (global.get $n$asm2wasm$import))
(global $nonZero (mut i32) (i32.const 1337))
(global $exportedNumber i32 (i32.const 42))
(export "big_negative" (func $big_negative))
(export "pick" (func $big_negative))
Expand Down Expand Up @@ -1238,6 +1239,12 @@
)
)
)
(global.set $nonZero
(i32.add
(global.get $nonZero)
(i32.const 1)
)
)
)
(func $vi (; 57 ;) (; has Stack IR ;) (param $0 i32)
(nop)
Expand Down
7 changes: 7 additions & 0 deletions test/unit.fromasm.clamp.no-opts
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
(global $tempDoublePtr (mut i32) (global.get $tempDoublePtr$asm2wasm$import))
(global $n (mut i32) (global.get $n$asm2wasm$import))
(global $STACKTOP (mut i32) (global.get $STACKTOP$asm2wasm$import))
(global $nonZero (mut i32) (i32.const 1337))
(global $exportedNumber i32 (i32.const 42))
(export "big_negative" (func $big_negative))
(export "pick" (func $exportMe))
Expand Down Expand Up @@ -2215,6 +2216,12 @@
(f64.const 9.78)
)
)
(global.set $nonZero
(i32.add
(global.get $nonZero)
(i32.const 1)
)
)
)
(func $v (; 83 ;)
(nop)
Expand Down
7 changes: 7 additions & 0 deletions test/unit.fromasm.imprecise
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
(global $Int (mut i32) (i32.const 0))
(global $Double (mut f64) (f64.const 0))
(global $n (mut i32) (global.get $n$asm2wasm$import))
(global $nonZero (mut i32) (i32.const 1337))
(global $exportedNumber i32 (i32.const 42))
(export "big_negative" (func $big_negative))
(export "pick" (func $big_negative))
Expand Down Expand Up @@ -1141,6 +1142,12 @@
(f64.const 1)
)
)
(global.set $nonZero
(i32.add
(global.get $nonZero)
(i32.const 1)
)
)
)
(func $vi (; 54 ;) (; has Stack IR ;) (param $0 i32)
(nop)
Expand Down
7 changes: 7 additions & 0 deletions test/unit.fromasm.imprecise.no-opts
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
(global $tempDoublePtr (mut i32) (global.get $tempDoublePtr$asm2wasm$import))
(global $n (mut i32) (global.get $n$asm2wasm$import))
(global $STACKTOP (mut i32) (global.get $STACKTOP$asm2wasm$import))
(global $nonZero (mut i32) (i32.const 1337))
(global $exportedNumber i32 (i32.const 42))
(export "big_negative" (func $big_negative))
(export "pick" (func $exportMe))
Expand Down Expand Up @@ -2100,6 +2101,12 @@
(f64.const 9.78)
)
)
(global.set $nonZero
(i32.add
(global.get $nonZero)
(i32.const 1)
)
)
)
(func $v (; 78 ;)
(nop)
Expand Down
Loading