From fcddb9ca819497790d4a3206edb763f7ce32d016 Mon Sep 17 00:00:00 2001 From: Alex Crichton Date: Wed, 15 Mar 2023 12:14:25 -0500 Subject: [PATCH] x64: Add lea-based lowering for iadd (#5986) * x64: Refactor `Amode` computation in ISLE This commit replaces the previous computation of `Amode` with a different set of rules that are intended to achieve the same purpose but are structured differently. The motivation for this commit is going to become more relevant in the next commit where `lea` will be used for the `iadd` instruction, possibly, on x64. When doing so it caused a stack overflow in the test suite during the compilation phase of a wasm module, namely as part of the `amode_add` function. This function is recursively defined in terms of itself and recurses as deep as the deepest `iadd`-chain in a program. A particular test in our test suite has a 10k-long chain of `iadd` which ended up causing a stack overflow in debug mode. This stack overflow is caused because the `amode_add` helper in ISLE unconditionally peels all the `iadd` nodes away and looks at all of them, even if most end up in intermediate registers along the way. Given that structure I couldn't find a way to easily abort the recursion. The new `to_amode` helper is structured in a similar fashion but attempts to instead only recurse far enough to fold items into the final `Amode` instead of recursing through items which themselves don't end up in the `Amode`. Put another way previously the `amode_add` helper might emit `x64_add` instructions, but it no longer does that. This goal of this commit is to preserve all the original `Amode` optimizations, however. For some parts, though, it relies more on egraph optimizations to run since if an `iadd` is 10k deep it doesn't try to find a constant buried 9k levels inside there to fold into the `Amode`. The hope, though, is that with egraphs having run already it's shuffled constants to the right most of the time and already folded any possible together. * x64: Add `lea`-based lowering for `iadd` This commit adds a rule for the lowering of `iadd` to use `lea` for 32 and 64-bit addition. The theoretical benefit of `lea` over the `add` instruction is that the `lea` variant can emulate a 3-operand instruction which doesn't destructively modify on of its operands. Additionally the `lea` operation can fold in other components such as constant additions and shifts. In practice, however, if `lea` is unconditionally used instead of `iadd` it ends up losing 10% performance on a local `meshoptimizer` benchmark. My best guess as to what's going on here is that my CPU's dedicated units for address computation are all overloaded while the ALUs are basically idle in a memory-intensive loop. Previously when the ALU was used for `add` and the address units for stores/loads it in theory pipelined things better (most of this is me shooting in the dark). To prevent the performance loss here I've updated the lowering of `iadd` to conditionally sometimes use `lea` and sometimes use `add` depending on how "complicated" the `Amode` is. Simple ones like `a + b` or `a + $imm` continue to use `add` (and its subsequent hypothetical extra `mov` necessary into the result). More complicated ones like `a + b + $imm` or `a + b << c + $imm` use `lea` as it can remove the need for extra instructions. Locally at least this fixes the performance loss relative to unconditionally using `lea`. One note is that this adds an `OperandSize` argument to the `MInst::LoadEffectiveAddress` variant to add an encoding for 32-bit `lea` in addition to the preexisting 64-bit encoding. * Conditionally use `lea` based on regalloc --- cranelift/codegen/src/isa/x64/inst.isle | 223 ++++---------- cranelift/codegen/src/isa/x64/inst/emit.rs | 86 +++++- cranelift/codegen/src/isa/x64/inst/mod.rs | 7 +- cranelift/codegen/src/isa/x64/lower.isle | 46 +-- cranelift/codegen/src/isa/x64/lower/isle.rs | 15 - cranelift/codegen/src/isle_prelude.rs | 5 + cranelift/codegen/src/prelude.isle | 5 + .../filetests/isa/x64/amode-opt.clif | 8 +- .../filetests/filetests/isa/x64/basic.clif | 6 +- .../filetests/filetests/isa/x64/branches.clif | 6 +- .../filetests/isa/x64/immediates.clif | 35 ++- .../filetests/filetests/isa/x64/lea.clif | 271 ++++++++++++++++++ .../filetests/filetests/isa/x64/load-op.clif | 6 +- .../filetests/isa/x64/pinned-reg.clif | 32 +-- .../filetests/filetests/isa/x64/table.clif | 18 +- .../filetests/isa/x64/uextend-elision.clif | 6 +- ...o_spectre_i32_access_0xffff0000_offset.wat | 32 +-- ...no_spectre_i8_access_0xffff0000_offset.wat | 32 +-- ...d_yes_spectre_i32_access_0x1000_offset.wat | 24 +- ...s_spectre_i32_access_0xffff0000_offset.wat | 44 +-- ...rd_yes_spectre_i8_access_0x1000_offset.wat | 24 +- ...es_spectre_i8_access_0xffff0000_offset.wat | 44 +-- ...o_spectre_i32_access_0xffff0000_offset.wat | 32 +-- ...no_spectre_i8_access_0xffff0000_offset.wat | 32 +-- ...d_yes_spectre_i32_access_0x1000_offset.wat | 24 +- ...s_spectre_i32_access_0xffff0000_offset.wat | 44 +-- ...rd_yes_spectre_i8_access_0x1000_offset.wat | 24 +- ...es_spectre_i8_access_0xffff0000_offset.wat | 44 +-- ...o_spectre_i32_access_0xffff0000_offset.wat | 16 +- ...no_spectre_i8_access_0xffff0000_offset.wat | 16 +- ...d_yes_spectre_i32_access_0x1000_offset.wat | 31 +- ...s_spectre_i32_access_0xffff0000_offset.wat | 38 +-- ...rd_yes_spectre_i8_access_0x1000_offset.wat | 31 +- ...es_spectre_i8_access_0xffff0000_offset.wat | 38 +-- ...o_spectre_i32_access_0xffff0000_offset.wat | 16 +- ...no_spectre_i8_access_0xffff0000_offset.wat | 16 +- ...d_yes_spectre_i32_access_0x1000_offset.wat | 31 +- ...s_spectre_i32_access_0xffff0000_offset.wat | 38 +-- ...rd_yes_spectre_i8_access_0x1000_offset.wat | 31 +- ...es_spectre_i8_access_0xffff0000_offset.wat | 38 +-- ...d_yes_spectre_i32_access_0x1000_offset.wat | 24 +- ...rd_yes_spectre_i8_access_0x1000_offset.wat | 24 +- ...rd_no_spectre_i32_access_0x1000_offset.wat | 2 +- ...ard_no_spectre_i8_access_0x1000_offset.wat | 2 +- ...d_yes_spectre_i32_access_0x1000_offset.wat | 2 +- ...rd_yes_spectre_i8_access_0x1000_offset.wat | 2 +- ...d_yes_spectre_i32_access_0x1000_offset.wat | 24 +- ...rd_yes_spectre_i8_access_0x1000_offset.wat | 24 +- ...d_yes_spectre_i32_access_0x1000_offset.wat | 24 +- ...rd_yes_spectre_i8_access_0x1000_offset.wat | 24 +- 50 files changed, 927 insertions(+), 740 deletions(-) create mode 100644 cranelift/filetests/filetests/isa/x64/lea.clif diff --git a/cranelift/codegen/src/isa/x64/inst.isle b/cranelift/codegen/src/isa/x64/inst.isle index faf5bc045279..1a0c04912c1c 100644 --- a/cranelift/codegen/src/isa/x64/inst.isle +++ b/cranelift/codegen/src/isa/x64/inst.isle @@ -170,7 +170,8 @@ ;; Loads the memory address of addr into dst. (LoadEffectiveAddress (addr SyntheticAmode) - (dst WritableGpr)) + (dst WritableGpr) + (size OperandSize)) ;; Sign-extended loads and moves: movs (bl bq wl wq lq) addr reg. (MovsxRmR (ext_mode ExtMode) @@ -990,25 +991,6 @@ ;; the given MachLabel. (RipRelative (target MachLabel)))) -;; Some Amode constructor helpers. - -(decl amode_with_flags (Amode MemFlags) Amode) -(extern constructor amode_with_flags amode_with_flags) - -(decl amode_imm_reg (u32 Gpr) Amode) -(extern constructor amode_imm_reg amode_imm_reg) - -(decl amode_imm_reg_flags (u32 Gpr MemFlags) Amode) -(rule (amode_imm_reg_flags offset base flags) - (amode_with_flags (amode_imm_reg offset base) flags)) - -(decl amode_imm_reg_reg_shift (u32 Gpr Gpr u8) Amode) -(extern constructor amode_imm_reg_reg_shift amode_imm_reg_reg_shift) - -(decl amode_imm_reg_reg_shift_flags (u32 Gpr Gpr u8 MemFlags) Amode) -(rule (amode_imm_reg_reg_shift_flags offset base index shift flags) - (amode_with_flags (amode_imm_reg_reg_shift offset base index shift) flags)) - ;; A helper to both check that the `Imm64` and `Offset32` values sum to less ;; than 32-bits AND return this summed `u32` value. Also, the `Imm64` will be ;; zero-extended from `Type` up to 64 bits. This is useful for `to_amode`. @@ -1017,168 +999,59 @@ ;;;; Amode lowering ;;;; -;; To generate an address for a memory access, we can pattern-match -;; various CLIF sub-trees to x64's complex addressing modes (`Amode`). -;; -;; Information about available addressing modes is available in -;; Intel's Software Developer's Manual, volume 2, section 2.1.5, -;; "Addressing-Mode Encoding of ModR/M and SIB Bytes." -;; -;; The general strategy to build an `Amode` is to traverse over the -;; input expression's addends, recursively deconstructing a tree of -;; `iadd` operators that add up parts of the address, updating the -;; `Amode` in an incremental fashion as we add in each piece. -;; -;; We start with an "immediate + register" form that encapsulates the -;; load/store's built-in `Offset32` and `invalid_reg` as the -;; register. This is given by `amode_initial`. Then we add `Value`s -;; one at a time with `amode_add`. (Why start with `invalid_reg` at -;; all? Because we don't want to special-case the first input and -;; duplicate rules; this lets us use the "add a value" logic even for -;; the first value.) -;; -;; It is always valid to use `amode_add` to add the one single -;; `address` input to the load/store (i.e., the `Value` given to -;; `to_amode`). In the fallback case, this is what we do. Then we get -;; an `Amode.ImmReg` with the `Offset32` and `Value` below and nothing -;; else; this always works and is not *that* bad. -;; -;; But we can often do better. The toplevel rule for `iadd` below will -;; turn an `(amode_add amode (iadd a b))` into two invocations of -;; `amode_add`, for each operand of the `iadd`. This is what allows us -;; to handle sums of many parts. -;; -;; Then we "just" need to work out how we can incorporate a new -;; component into an existing addressing mode: -;; -;; - Case 1: When we have an `ImmReg` and the register is -;; `invalid_reg` (the initial `Amode` above), we can put the new -;; addend into a register and insert it into the `ImmReg`. -;; -;; - Case 2: When we have an `ImmReg` with a valid register already, -;; and we have another register to add, we can transition to an -;; `ImmRegRegShift`. -;; -;; - Case 3: When we're adding an `ishl`, we can refine the above rule -;; and use the built-in multiplier of 1, 2, 4, 8 to implement a -;; left-shift by 0, 1, 2, 3. -;; -;; - Case 4: When we are adding another constant offset, we can fold -;; it into the existing offset, as long as the sum still fits into -;; the signed 32-bit field. -;; -;; - Case 5: And as a general fallback, we can generate a new `add` -;; instruction and add the new addend to an existing component of -;; the `Amode`. +;; Converts a `Value` and a static offset into an `Amode` for x64, attempting +;; to be as fancy as possible with offsets/registers/shifts/etc to make maximal +;; use of the x64 addressing modes. (decl to_amode (MemFlags Value Offset32) Amode) -;; Initial step in amode processing: create an ImmReg with -;; (invalid_reg) and encapsulating the flags and offset from the -;; load/store. -(decl amode_initial (MemFlags Offset32) Amode) -(rule (amode_initial flags (offset32 off)) - (Amode.ImmReg off (invalid_reg) flags)) +;; Base case, "just put it in a register" +(rule (to_amode flags base offset) + (Amode.ImmReg offset base flags)) -;; One step in amode processing: take an existing amode and add -;; another value to it. -(decl amode_add (Amode Value) Amode) +;; Slightly-more-fancy case, if the address is the addition of two things then +;; delegate to the `to_amode_add` helper. +(rule 1 (to_amode flags (iadd x y) offset) + (to_amode_add flags x y offset)) -;; -- Top-level driver: pull apart the addends. -;; -;; Any amode can absorb an `iadd` by absorbing first the LHS of the -;; add, then the RHS. -;; -;; Priority 2 to take this above fallbacks and ensure we traverse the -;; `iadd` tree fully. -(rule 2 (amode_add amode (iadd x y)) - (let ((amode1 Amode (amode_add amode x)) - (amode2 Amode (amode_add amode1 y))) - amode2)) - -;; -- Case 1 (adding a register to the initial Amode with invalid_reg). -;; -;; An Amode.ImmReg with invalid_reg (initial state) can absorb a -;; register as the base register. -(rule (amode_add (Amode.ImmReg off (invalid_reg) flags) value) - (Amode.ImmReg off value flags)) +;; Same as `to_amode`, except that the base address is computed via the addition +;; of the two `Value` arguments provided. +(decl to_amode_add (MemFlags Value Value Offset32) Amode) -;; -- Case 2 (adding a register to an Amode with a register already). -;; -;; An Amode.ImmReg can absorb another register as the index register. -(rule (amode_add (Amode.ImmReg off (valid_reg base) flags) value) - ;; Shift of 0 --> base + 1*value. - (Amode.ImmRegRegShift off base value 0 flags)) +;; Base case, "just put things in registers". Note that the shift value of 0 +;; here means `x + (y << 0)` which is the same as `x + y`. +(rule (to_amode_add flags x y offset) + (Amode.ImmRegRegShift offset x y 0 flags)) -;; -- Case 3 (adding a shifted value to an Amode). -;; -;; An Amode.ImmReg can absorb a shift of another register as the index register. -;; -;; Priority 2 to take these rules above generic case. -(rule 2 (amode_add (Amode.ImmReg off (valid_reg base) flags) (ishl index (iconst (uimm8 shift)))) +;; If the one of the arguments being added is itself a constant shift then +;; that can be modeled directly so long as the shift is a modestly small amount. +(rule 1 (to_amode_add flags x (ishl y (iconst (uimm8 shift))) offset) + (if (u32_lteq (u8_as_u32 shift) 3)) + (Amode.ImmRegRegShift offset x y shift flags)) +(rule 2 (to_amode_add flags (ishl y (iconst (uimm8 shift))) x offset) (if (u32_lteq (u8_as_u32 shift) 3)) - (Amode.ImmRegRegShift off base index shift flags)) + (Amode.ImmRegRegShift offset x y shift flags)) -;; -- Case 4 (absorbing constant offsets). -;; -;; An Amode can absorb a constant (i64, or extended i32) as long as -;; the sum still fits in the signed-32-bit offset. +;; Constant extraction rules. ;; -;; Priority 3 in order to take this option above the fallback -;; (immediate in register). Two rules, for imm+reg and -;; imm+reg+scale*reg cases. -(rule 3 (amode_add (Amode.ImmReg off base flags) - (iconst (simm32 c))) - (if-let sum (s32_add_fallible off c)) - (Amode.ImmReg sum base flags)) -(rule 3 (amode_add (Amode.ImmRegRegShift off base index shift flags) - (iconst (simm32 c))) - (if-let sum (s32_add_fallible off c)) - (Amode.ImmRegRegShift sum base index shift flags)) - -;; Likewise for a zero-extended i32 const, as long as the constant -;; wasn't negative. (Why nonnegative? Because adding a -;; non-sign-extended negative to a 64-bit address is not the same as -;; adding in simm32-space.) -(rule 3 (amode_add (Amode.ImmReg off base flags) - (uextend (iconst (simm32 (u32_nonnegative c))))) - (if-let sum (s32_add_fallible off c)) - (Amode.ImmReg sum base flags)) -(rule 3 (amode_add (Amode.ImmRegRegShift off base index shift flags) - (uextend (iconst (simm32 (u32_nonnegative c))))) - (if-let sum (s32_add_fallible off c)) - (Amode.ImmRegRegShift sum base index shift flags)) - -;; Likewise for a sign-extended i32 const. -(rule 3 (amode_add (Amode.ImmReg off base flags) - (sextend (iconst (simm32 c)))) - (if-let sum (s32_add_fallible off c)) - (Amode.ImmReg sum base flags)) -(rule 3 (amode_add (Amode.ImmRegRegShift off base index shift flags) - (sextend (iconst (simm32 c)))) - (if-let sum (s32_add_fallible off c)) - (Amode.ImmRegRegShift sum base index shift flags)) - -;; -- Case 5 (fallback to add a new value to an imm+reg+scale*reg). +;; These rules attempt to find a constant within one of `x` or `y`, or deeper +;; within them if they have their own adds. These only succeed if the constant +;; itself can be represented with 32-bits and can be infallibly added to the +;; offset that we already have. ;; -;; An Amode.ImmRegRegShift can absorb any other value by creating a -;; new add instruction and replacing the base with -;; (base+value). -(rule (amode_add (Amode.ImmRegRegShift off base index shift flags) value) - (let ((sum Gpr (x64_add $I64 base value))) - (Amode.ImmRegRegShift off sum index shift flags))) - -;; Finally, define the toplevel `to_amode`. -(rule (to_amode flags base @ (value_type (ty_addr64 _)) offset) - (amode_finalize (amode_add (amode_initial flags offset) base))) - -;; If an amode has no registers at all and only offsets (a constant -;; value), we need to "finalize" it by sticking in a zero'd reg in -;; place of the (invalid_reg) produced by (amode_initial). -(decl amode_finalize (Amode) Amode) -(rule 1 (amode_finalize (Amode.ImmReg off (invalid_reg) flags)) - (Amode.ImmReg off (imm $I64 0) flags)) -(rule 0 (amode_finalize amode) - amode) +;; Note the recursion here where this rule is defined in terms of itself to +;; "peel" layers of constants. +(rule 3 (to_amode_add flags (iadd x (iconst (simm32 c))) y offset) + (if-let sum (s32_add_fallible offset c)) + (to_amode_add flags x y sum)) +(rule 4 (to_amode_add flags x (iadd y (iconst (simm32 c))) offset) + (if-let sum (s32_add_fallible offset c)) + (to_amode_add flags x y sum)) +(rule 5 (to_amode_add flags x (iconst (simm32 c)) offset) + (if-let sum (s32_add_fallible offset c)) + (to_amode flags x sum)) +(rule 6 (to_amode_add flags (iconst (simm32 c)) x offset) + (if-let sum (s32_add_fallible offset c)) + (to_amode flags x sum)) ;; Offsetting an Amode. Used when we need to do consecutive ;; loads/stores to adjacent addresses. @@ -3787,10 +3660,10 @@ (inst MInst (MInst.Neg size src dst))) (ProducesFlags.ProducesFlagsReturnsResultWithConsumer inst dst))) -(decl x64_lea (SyntheticAmode) Gpr) -(rule (x64_lea addr) +(decl x64_lea (Type SyntheticAmode) Gpr) +(rule (x64_lea ty addr) (let ((dst WritableGpr (temp_writable_gpr)) - (_ Unit (emit (MInst.LoadEffectiveAddress addr dst)))) + (_ Unit (emit (MInst.LoadEffectiveAddress addr dst (operand_size_of_type_32_64 ty))))) dst)) ;; Helper for creating `ud2` instructions. diff --git a/cranelift/codegen/src/isa/x64/inst/emit.rs b/cranelift/codegen/src/isa/x64/inst/emit.rs index 4e409979dddf..d8718ad8e537 100644 --- a/cranelift/codegen/src/isa/x64/inst/emit.rs +++ b/cranelift/codegen/src/isa/x64/inst/emit.rs @@ -869,20 +869,84 @@ pub(crate) fn emit( ) } - Inst::LoadEffectiveAddress { addr, dst } => { + Inst::LoadEffectiveAddress { addr, dst, size } => { let dst = allocs.next(dst.to_reg().to_reg()); let amode = addr.finalize(state, sink).with_allocs(allocs); - emit_std_reg_mem( - sink, - LegacyPrefixes::None, - 0x8D, - 1, - dst, - &amode, - RexFlags::set_w(), - 0, - ); + // If this `lea` can actually get encoded as an `add` then do that + // instead. Currently all candidate `iadd`s become an `lea` + // pseudo-instruction here but maximizing the sue of `lea` is not + // necessarily optimal. The `lea` instruction goes through dedicated + // address units on cores which are finite and disjoint from the + // general ALU, so if everything uses `lea` then those units can get + // saturated while leaving the ALU idle. + // + // To help make use of more parts of a cpu, this attempts to use + // `add` when it's semantically equivalent to `lea`, or otherwise + // when the `dst` register is the same as the `base` or `index` + // register. + // + // FIXME: ideally regalloc is informed of this constraint. Register + // allocation of `lea` should "attempt" to put the `base` in the + // same register as `dst` but not at the expense of generating a + // `mov` instruction. Currently that's not possible but perhaps one + // day it may be worth it. + match amode { + // If `base == dst` then this is `add $imm, %dst`, so encode + // that instead. + Amode::ImmReg { + simm32, + base, + flags: _, + } if base == dst => { + let inst = Inst::alu_rmi_r( + *size, + AluRmiROpcode::Add, + RegMemImm::imm(simm32), + Writable::from_reg(dst), + ); + inst.emit(&[], sink, info, state); + } + // If the offset is 0 and the shift is 0 (meaning multiplication + // by 1) then: + // + // * If `base == dst`, then this is `add %index, %base` + // * If `index == dst`, then this is `add %base, %index` + // + // Encode the appropriate instruction here in that case. + Amode::ImmRegRegShift { + simm32: 0, + base, + index, + shift: 0, + flags: _, + } if base == dst || index == dst => { + let (dst, operand) = if base == dst { + (base, index) + } else { + (index, base) + }; + let inst = Inst::alu_rmi_r( + *size, + AluRmiROpcode::Add, + RegMemImm::reg(operand.to_reg()), + Writable::from_reg(dst.to_reg()), + ); + inst.emit(&[], sink, info, state); + } + + // If `lea`'s 3-operand mode is leveraged by regalloc, or if + // it's fancy like imm-plus-shift-plus-base, then `lea` is + // actually emitted. + _ => { + let flags = match size { + OperandSize::Size32 => RexFlags::clear_w(), + OperandSize::Size64 => RexFlags::set_w(), + _ => unreachable!(), + }; + emit_std_reg_mem(sink, LegacyPrefixes::None, 0x8D, 1, dst, &amode, flags, 0); + } + }; } Inst::MovsxRmR { ext_mode, src, dst } => { diff --git a/cranelift/codegen/src/isa/x64/inst/mod.rs b/cranelift/codegen/src/isa/x64/inst/mod.rs index b7865b21b54d..77689c6424c2 100644 --- a/cranelift/codegen/src/isa/x64/inst/mod.rs +++ b/cranelift/codegen/src/isa/x64/inst/mod.rs @@ -429,6 +429,7 @@ impl Inst { Inst::LoadEffectiveAddress { addr: addr.into(), dst: WritableGpr::from_writable_reg(dst).unwrap(), + size: OperandSize::Size64, } } @@ -1432,8 +1433,8 @@ impl PrettyPrint for Inst { format!("{} {}, {}", ljustify("movq".to_string()), src, dst) } - Inst::LoadEffectiveAddress { addr, dst } => { - let dst = pretty_print_reg(dst.to_reg().to_reg(), 8, allocs); + Inst::LoadEffectiveAddress { addr, dst, size } => { + let dst = pretty_print_reg(dst.to_reg().to_reg(), size.to_bytes(), allocs); let addr = addr.pretty_print(8, allocs); format!("{} {}, {}", ljustify("lea".to_string()), addr, dst) } @@ -2173,7 +2174,7 @@ fn x64_get_operands VReg>(inst: &Inst, collector: &mut OperandCol collector.reg_def(dst.to_writable_reg()); src.get_operands(collector); } - Inst::LoadEffectiveAddress { addr: src, dst } => { + Inst::LoadEffectiveAddress { addr: src, dst, .. } => { collector.reg_def(dst.to_writable_reg()); src.get_operands(collector); } diff --git a/cranelift/codegen/src/isa/x64/lower.isle b/cranelift/codegen/src/isa/x64/lower.isle index b5fcffe4804c..715b6079f0d7 100644 --- a/cranelift/codegen/src/isa/x64/lower.isle +++ b/cranelift/codegen/src/isa/x64/lower.isle @@ -41,17 +41,27 @@ ;; `i64` and smaller. -;; Add two registers. -(rule -5 (lower (has_type (fits_in_64 ty) +;; Base case for 8 and 16-bit types +(rule -6 (lower (has_type (fits_in_16 ty) (iadd x y))) (x64_add ty x y)) -;; The above case handles when the rhs is an immediate or a sinkable load, but -;; additionally add lhs meets these criteria. - +;; Base case for 32 and 64-bit types which might end up using the `lea` +;; instruction to fold multiple operations into one. +;; +;; Note that at this time this always generates a `lea` pseudo-instruction, +;; but the actual instruction emitted might be an `add` if it's equivalent. +;; For more details on this see the `emit.rs` logic to emit +;; `LoadEffectiveAddress`. +(rule -5 (lower (has_type (ty_32_or_64 ty) (iadd x y))) + (x64_lea ty (to_amode_add (mem_flags_trusted) x y (zero_offset)))) + +;; Higher-priority cases than the previous two where a load can be sunk into +;; the add instruction itself. Note that both operands are tested for +;; sink-ability since addition is commutative (rule -4 (lower (has_type (fits_in_64 ty) - (iadd (simm32_from_value x) y))) - (x64_add ty y x)) + (iadd x (sinkable_load y)))) + (x64_add ty x y)) (rule -3 (lower (has_type (fits_in_64 ty) (iadd (sinkable_load x) y))) (x64_add ty y x)) @@ -442,13 +452,14 @@ (extern constructor ishl_i8x16_mask_table ishl_i8x16_mask_table) (rule (ishl_i8x16_mask (RegMemImm.Reg amt)) (let ((mask_table SyntheticAmode (ishl_i8x16_mask_table)) - (base_mask_addr Gpr (x64_lea mask_table)) + (base_mask_addr Gpr (x64_lea $I64 mask_table)) (mask_offset Gpr (x64_shl $I64 amt (imm8_to_imm8_gpr 4)))) - (amode_imm_reg_reg_shift 0 - base_mask_addr - mask_offset - 0))) + (Amode.ImmRegRegShift 0 + base_mask_addr + mask_offset + 0 + (mem_flags_trusted)))) (rule (ishl_i8x16_mask (RegMemImm.Mem amt)) (ishl_i8x16_mask (RegMemImm.Reg (x64_load $I64 amt (ExtKind.None))))) @@ -546,14 +557,15 @@ (extern constructor ushr_i8x16_mask_table ushr_i8x16_mask_table) (rule (ushr_i8x16_mask (RegMemImm.Reg amt)) (let ((mask_table SyntheticAmode (ushr_i8x16_mask_table)) - (base_mask_addr Gpr (x64_lea mask_table)) + (base_mask_addr Gpr (x64_lea $I64 mask_table)) (mask_offset Gpr (x64_shl $I64 amt (imm8_to_imm8_gpr 4)))) - (amode_imm_reg_reg_shift 0 - base_mask_addr - mask_offset - 0))) + (Amode.ImmRegRegShift 0 + base_mask_addr + mask_offset + 0 + (mem_flags_trusted)))) (rule (ushr_i8x16_mask (RegMemImm.Mem amt)) (ushr_i8x16_mask (RegMemImm.Reg (x64_load $I64 amt (ExtKind.None))))) diff --git a/cranelift/codegen/src/isa/x64/lower/isle.rs b/cranelift/codegen/src/isa/x64/lower/isle.rs index 4d0a960151fb..8e297e02c0cb 100644 --- a/cranelift/codegen/src/isa/x64/lower/isle.rs +++ b/cranelift/codegen/src/isa/x64/lower/isle.rs @@ -315,21 +315,6 @@ impl Context for IsleContext<'_, '_, MInst, X64Backend> { RegMem::mem(addr.clone()) } - #[inline] - fn amode_imm_reg_reg_shift(&mut self, simm32: u32, base: Gpr, index: Gpr, shift: u8) -> Amode { - Amode::imm_reg_reg_shift(simm32, base, index, shift) - } - - #[inline] - fn amode_imm_reg(&mut self, simm32: u32, base: Gpr) -> Amode { - Amode::imm_reg(simm32, base.to_reg()) - } - - #[inline] - fn amode_with_flags(&mut self, amode: &Amode, flags: MemFlags) -> Amode { - amode.with_flags(flags) - } - #[inline] fn amode_to_synthetic_amode(&mut self, amode: &Amode) -> SyntheticAmode { amode.clone().into() diff --git a/cranelift/codegen/src/isle_prelude.rs b/cranelift/codegen/src/isle_prelude.rs index 9d78044b30dc..cf0caac323b3 100644 --- a/cranelift/codegen/src/isle_prelude.rs +++ b/cranelift/codegen/src/isle_prelude.rs @@ -648,6 +648,11 @@ macro_rules! isle_common_prelude_methods { offset as u32 } + #[inline] + fn u32_to_offset32(&mut self, offset: u32) -> Offset32 { + Offset32::new(offset as i32) + } + fn range(&mut self, start: usize, end: usize) -> Range { (start, end) } diff --git a/cranelift/codegen/src/prelude.isle b/cranelift/codegen/src/prelude.isle index 4815cf63577a..ef2741570a83 100644 --- a/cranelift/codegen/src/prelude.isle +++ b/cranelift/codegen/src/prelude.isle @@ -439,6 +439,10 @@ (decl pure offset32_to_u32 (Offset32) u32) (extern constructor offset32_to_u32 offset32_to_u32) +;; Convert a number to an `Offset32` +(decl pure u32_to_offset32 (u32) Offset32) +(extern constructor u32_to_offset32 u32_to_offset32) + ;; This is a direct import of `IntCC::unsigned`. ;; Get the corresponding IntCC with the signed component removed. ;; For conditions without a signed component, this is a no-op. @@ -491,3 +495,4 @@ ;;;; Automatic conversions ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; (convert Offset32 u32 offset32_to_u32) +(convert u32 Offset32 u32_to_offset32) diff --git a/cranelift/filetests/filetests/isa/x64/amode-opt.clif b/cranelift/filetests/filetests/isa/x64/amode-opt.clif index eef1d2893313..e3d689998897 100644 --- a/cranelift/filetests/filetests/isa/x64/amode-opt.clif +++ b/cranelift/filetests/filetests/isa/x64/amode-opt.clif @@ -1,4 +1,6 @@ test compile precise-output +set use_egraphs=true +set opt_level=speed target x86_64 function %amode_add(i64, i64) -> i64 { @@ -244,8 +246,7 @@ block0(v0: i64, v1: i32, v2: i32): ; pushq %rbp ; movq %rsp, %rbp ; block0: -; movq %rsi, %r8 -; addl %r8d, %edx, %r8d +; lea 0(%rsi,%rdx,1), %r8d ; shll $2, %r8d, %r8d ; movq -1(%rdi,%r8,1), %rax ; movq %rbp, %rsp @@ -257,8 +258,7 @@ block0(v0: i64, v1: i32, v2: i32): ; pushq %rbp ; movq %rsp, %rbp ; block1: ; offset 0x4 -; movq %rsi, %r8 -; addl %edx, %r8d +; leal (%rsi, %rdx), %r8d ; shll $2, %r8d ; movq -1(%rdi, %r8), %rax ; trap: heap_oob ; movq %rbp, %rsp diff --git a/cranelift/filetests/filetests/isa/x64/basic.clif b/cranelift/filetests/filetests/isa/x64/basic.clif index d10885cd0714..01fac2deb86d 100644 --- a/cranelift/filetests/filetests/isa/x64/basic.clif +++ b/cranelift/filetests/filetests/isa/x64/basic.clif @@ -11,8 +11,7 @@ block0(v0: i32, v1: i32): ; pushq %rbp ; movq %rsp, %rbp ; block0: -; movq %rdi, %rax -; addl %eax, %esi, %eax +; lea 0(%rdi,%rsi,1), %eax ; movq %rbp, %rsp ; popq %rbp ; ret @@ -22,8 +21,7 @@ block0(v0: i32, v1: i32): ; pushq %rbp ; movq %rsp, %rbp ; block1: ; offset 0x4 -; movq %rdi, %rax -; addl %esi, %eax +; leal (%rdi, %rsi), %eax ; movq %rbp, %rsp ; popq %rbp ; retq diff --git a/cranelift/filetests/filetests/isa/x64/branches.clif b/cranelift/filetests/filetests/isa/x64/branches.clif index 01d9c3abdc1d..bdaf6b5b614a 100644 --- a/cranelift/filetests/filetests/isa/x64/branches.clif +++ b/cranelift/filetests/filetests/isa/x64/branches.clif @@ -784,8 +784,7 @@ block5(v5: i32): ; movl $4, %esi ; jmp label7 ; block7: -; movq %rdi, %rax -; addl %eax, %esi, %eax +; lea 0(%rdi,%rsi,1), %eax ; movq %rbp, %rsp ; popq %rbp ; ret @@ -825,8 +824,7 @@ block5(v5: i32): ; block6: ; offset 0x59 ; movl $4, %esi ; block7: ; offset 0x5e -; movq %rdi, %rax -; addl %esi, %eax +; leal (%rdi, %rsi), %eax ; movq %rbp, %rsp ; popq %rbp ; retq diff --git a/cranelift/filetests/filetests/isa/x64/immediates.clif b/cranelift/filetests/filetests/isa/x64/immediates.clif index bdbd7ece6ab4..62528f0b6dcf 100644 --- a/cranelift/filetests/filetests/isa/x64/immediates.clif +++ b/cranelift/filetests/filetests/isa/x64/immediates.clif @@ -19,15 +19,15 @@ block0(v0: i64, v1: i64): ; pushq %rbp ; movq %rsp, %rbp ; block0: -; movq %rdi, %r9 -; addq %r9, const(0), %r9 -; movq %r9, 0(%rsi) -; movq %rdi, %r10 -; subq %r10, const(0), %r10 -; movq %r10, 0(%rsi) +; movabsq $-18765284782900, %r9 +; lea 0(%rdi,%r9,1), %r11 +; movq %r11, 0(%rsi) ; movq %rdi, %r11 -; andq %r11, const(0), %r11 +; subq %r11, const(0), %r11 ; movq %r11, 0(%rsi) +; movq %rdi, %rax +; andq %rax, const(0), %rax +; movq %rax, 0(%rsi) ; orq %rdi, const(0), %rdi ; movq %rdi, 0(%rsi) ; movq %rbp, %rsp @@ -39,24 +39,21 @@ block0(v0: i64, v1: i64): ; pushq %rbp ; movq %rsp, %rbp ; block1: ; offset 0x4 -; movq %rdi, %r9 -; addq 0x32(%rip), %r9 -; movq %r9, (%rsi) ; trap: heap_oob -; movq %rdi, %r10 -; subq 0x25(%rip), %r10 -; movq %r10, (%rsi) ; trap: heap_oob +; movabsq $18446725308424768716, %r9 +; leaq (%rdi, %r9), %r11 +; movq %r11, (%rsi) ; trap: heap_oob ; movq %rdi, %r11 -; andq 0x18(%rip), %r11 +; subq 0x20(%rip), %r11 ; movq %r11, (%rsi) ; trap: heap_oob -; orq 0xe(%rip), %rdi +; movq %rdi, %rax +; andq 0x13(%rip), %rax +; movq %rax, (%rsi) ; trap: heap_oob +; orq 9(%rip), %rdi ; movq %rdi, (%rsi) ; trap: heap_oob ; movq %rbp, %rsp ; popq %rbp ; retq -; addb %al, (%rax) -; addb %al, (%rax) -; addb %al, (%rax) -; int3 +; addb %cl, %ah ; int3 ; fstp %st(5) ; outb %al, %dx diff --git a/cranelift/filetests/filetests/isa/x64/lea.clif b/cranelift/filetests/filetests/isa/x64/lea.clif new file mode 100644 index 000000000000..4dc8182bd1ec --- /dev/null +++ b/cranelift/filetests/filetests/isa/x64/lea.clif @@ -0,0 +1,271 @@ +test compile precise-output +target x86_64 + +function %add_i32(i32, i32) -> i32 { +block0(v0: i32, v1: i32): + v2 = iadd v0, v1 + return v2 +} + +; VCode: +; pushq %rbp +; movq %rsp, %rbp +; block0: +; lea 0(%rdi,%rsi,1), %eax +; movq %rbp, %rsp +; popq %rbp +; ret +; +; Disassembled: +; block0: ; offset 0x0 +; pushq %rbp +; movq %rsp, %rbp +; block1: ; offset 0x4 +; leal (%rdi, %rsi), %eax +; movq %rbp, %rsp +; popq %rbp +; retq + +function %add_i64(i64, i64) -> i64 { +block0(v0: i64, v1: i64): + v2 = iadd v0, v1 + return v2 +} + +; VCode: +; pushq %rbp +; movq %rsp, %rbp +; block0: +; lea 0(%rdi,%rsi,1), %rax +; movq %rbp, %rsp +; popq %rbp +; ret +; +; Disassembled: +; block0: ; offset 0x0 +; pushq %rbp +; movq %rsp, %rbp +; block1: ; offset 0x4 +; leaq (%rdi, %rsi), %rax +; movq %rbp, %rsp +; popq %rbp +; retq + +function %add_i32_const(i32) -> i32 { +block0(v0: i32): + v1 = iconst.i32 100 + v2 = iadd v0, v1 + return v2 +} + +; VCode: +; pushq %rbp +; movq %rsp, %rbp +; block0: +; lea 100(%rdi), %eax +; movq %rbp, %rsp +; popq %rbp +; ret +; +; Disassembled: +; block0: ; offset 0x0 +; pushq %rbp +; movq %rsp, %rbp +; block1: ; offset 0x4 +; leal 0x64(%rdi), %eax +; movq %rbp, %rsp +; popq %rbp +; retq + +function %add_i64_const(i64) -> i64 { +block0(v0: i64): + v1 = iconst.i64 100 + v2 = iadd v0, v1 + return v2 +} + +; VCode: +; pushq %rbp +; movq %rsp, %rbp +; block0: +; lea 100(%rdi), %rax +; movq %rbp, %rsp +; popq %rbp +; ret +; +; Disassembled: +; block0: ; offset 0x0 +; pushq %rbp +; movq %rsp, %rbp +; block1: ; offset 0x4 +; leaq 0x64(%rdi), %rax +; movq %rbp, %rsp +; popq %rbp +; retq + +function %add_i32_i32_const(i32, i32) -> i32 { +block0(v0: i32, v1: i32): + v2 = iconst.i32 100 + v3 = iadd v0, v1 + v4 = iadd v3, v2 + return v4 +} + +; VCode: +; pushq %rbp +; movq %rsp, %rbp +; block0: +; lea 100(%rdi,%rsi,1), %eax +; movq %rbp, %rsp +; popq %rbp +; ret +; +; Disassembled: +; block0: ; offset 0x0 +; pushq %rbp +; movq %rsp, %rbp +; block1: ; offset 0x4 +; leal 0x64(%rdi, %rsi), %eax +; movq %rbp, %rsp +; popq %rbp +; retq + +function %add_i64_i64_const(i64, i64) -> i64 { +block0(v0: i64, v1: i64): + v2 = iconst.i64 100 + v3 = iadd v0, v1 + v4 = iadd v3, v2 + return v4 +} + +; VCode: +; pushq %rbp +; movq %rsp, %rbp +; block0: +; lea 100(%rdi,%rsi,1), %rax +; movq %rbp, %rsp +; popq %rbp +; ret +; +; Disassembled: +; block0: ; offset 0x0 +; pushq %rbp +; movq %rsp, %rbp +; block1: ; offset 0x4 +; leaq 0x64(%rdi, %rsi), %rax +; movq %rbp, %rsp +; popq %rbp +; retq + +function %add_i32_i32_mul_const(i32, i32) -> i32 { +block0(v0: i32, v1: i32): + v2 = iconst.i32 100 + v3 = iconst.i32 2 + v4 = ishl v1, v3 + v5 = iadd v0, v4 + v6 = iadd v5, v2 + return v6 +} + +; VCode: +; pushq %rbp +; movq %rsp, %rbp +; block0: +; lea 100(%rdi,%rsi,4), %eax +; movq %rbp, %rsp +; popq %rbp +; ret +; +; Disassembled: +; block0: ; offset 0x0 +; pushq %rbp +; movq %rsp, %rbp +; block1: ; offset 0x4 +; leal 0x64(%rdi, %rsi, 4), %eax +; movq %rbp, %rsp +; popq %rbp +; retq + +function %add_i64_i64_mul_const(i64, i64) -> i64 { +block0(v0: i64, v1: i64): + v2 = iconst.i64 100 + v3 = iconst.i64 2 + v4 = ishl v1, v3 + v5 = iadd v0, v4 + v6 = iadd v5, v2 + return v6 +} + +; VCode: +; pushq %rbp +; movq %rsp, %rbp +; block0: +; lea 100(%rdi,%rsi,4), %rax +; movq %rbp, %rsp +; popq %rbp +; ret +; +; Disassembled: +; block0: ; offset 0x0 +; pushq %rbp +; movq %rsp, %rbp +; block1: ; offset 0x4 +; leaq 0x64(%rdi, %rsi, 4), %rax +; movq %rbp, %rsp +; popq %rbp +; retq + +function %add_i32_i32_mul(i32, i32) -> i32 { +block0(v0: i32, v1: i32): + v3 = iconst.i32 2 + v4 = ishl v1, v3 + v5 = iadd v0, v4 + return v5 +} + +; VCode: +; pushq %rbp +; movq %rsp, %rbp +; block0: +; lea 0(%rdi,%rsi,4), %eax +; movq %rbp, %rsp +; popq %rbp +; ret +; +; Disassembled: +; block0: ; offset 0x0 +; pushq %rbp +; movq %rsp, %rbp +; block1: ; offset 0x4 +; leal (%rdi, %rsi, 4), %eax +; movq %rbp, %rsp +; popq %rbp +; retq + +function %add_i64_i64_mul(i64, i64) -> i64 { +block0(v0: i64, v1: i64): + v3 = iconst.i64 2 + v4 = ishl v1, v3 + v5 = iadd v0, v4 + return v5 +} + +; VCode: +; pushq %rbp +; movq %rsp, %rbp +; block0: +; lea 0(%rdi,%rsi,4), %rax +; movq %rbp, %rsp +; popq %rbp +; ret +; +; Disassembled: +; block0: ; offset 0x0 +; pushq %rbp +; movq %rsp, %rbp +; block1: ; offset 0x4 +; leaq (%rdi, %rsi, 4), %rax +; movq %rbp, %rsp +; popq %rbp +; retq + diff --git a/cranelift/filetests/filetests/isa/x64/load-op.clif b/cranelift/filetests/filetests/isa/x64/load-op.clif index ab140cb5fefd..3c4e5fde972c 100644 --- a/cranelift/filetests/filetests/isa/x64/load-op.clif +++ b/cranelift/filetests/filetests/isa/x64/load-op.clif @@ -155,8 +155,7 @@ block0(v0: i64, v1: i64): ; movq %rsp, %rbp ; block0: ; movq 0(%rdi), %r8 -; movq %r8, %r9 -; addq %r9, %rdi, %r9 +; lea 0(%r8,%rdi,1), %r9 ; movq %r9, 0(%rsi) ; movq 0(%r8,%rdi,1), %rax ; movq %rbp, %rsp @@ -169,8 +168,7 @@ block0(v0: i64, v1: i64): ; movq %rsp, %rbp ; block1: ; offset 0x4 ; movq (%rdi), %r8 ; trap: heap_oob -; movq %r8, %r9 -; addq %rdi, %r9 +; leaq (%r8, %rdi), %r9 ; movq %r9, (%rsi) ; trap: heap_oob ; movq (%r8, %rdi), %rax ; trap: heap_oob ; movq %rbp, %rsp diff --git a/cranelift/filetests/filetests/isa/x64/pinned-reg.clif b/cranelift/filetests/filetests/isa/x64/pinned-reg.clif index 26bc0b280fbe..36797fb117ac 100644 --- a/cranelift/filetests/filetests/isa/x64/pinned-reg.clif +++ b/cranelift/filetests/filetests/isa/x64/pinned-reg.clif @@ -14,9 +14,9 @@ block0: ; pushq %rbp ; movq %rsp, %rbp ; block0: -; movq %r15, %rsi -; addq %rsi, $1, %rsi -; movq %rsi, %r15 +; movq %r15, %rdi +; lea 1(%rdi), %rdi +; movq %rdi, %r15 ; movq %rbp, %rsp ; popq %rbp ; ret @@ -26,9 +26,9 @@ block0: ; pushq %rbp ; movq %rsp, %rbp ; block1: ; offset 0x4 -; movq %r15, %rsi -; addq $1, %rsi -; movq %rsi, %r15 +; movq %r15, %rdi +; addq $1, %rdi +; movq %rdi, %r15 ; movq %rbp, %rsp ; popq %rbp ; retq @@ -45,12 +45,12 @@ block0: ; pushq %rbp ; movq %rsp, %rbp ; subq %rsp, $16, %rsp -; movq %rsi, 0(%rsp) +; movq %rdi, 0(%rsp) ; block0: -; movq %r15, %rsi -; addq %rsi, $1, %rsi -; movq %rsi, %r15 -; movq 0(%rsp), %rsi +; movq %r15, %rdi +; lea 1(%rdi), %rdi +; movq %rdi, %r15 +; movq 0(%rsp), %rdi ; addq %rsp, $16, %rsp ; movq %rbp, %rsp ; popq %rbp @@ -61,12 +61,12 @@ block0: ; pushq %rbp ; movq %rsp, %rbp ; subq $0x10, %rsp -; movq %rsi, (%rsp) +; movq %rdi, (%rsp) ; block1: ; offset 0xc -; movq %r15, %rsi -; addq $1, %rsi -; movq %rsi, %r15 -; movq (%rsp), %rsi +; movq %r15, %rdi +; addq $1, %rdi +; movq %rdi, %r15 +; movq (%rsp), %rdi ; addq $0x10, %rsp ; movq %rbp, %rsp ; popq %rbp diff --git a/cranelift/filetests/filetests/isa/x64/table.clif b/cranelift/filetests/filetests/isa/x64/table.clif index 101debb9be53..0bfe7a35fd32 100644 --- a/cranelift/filetests/filetests/isa/x64/table.clif +++ b/cranelift/filetests/filetests/isa/x64/table.clif @@ -26,11 +26,10 @@ block0(v0: i32, v1: r64, v2: i64): ; block1: ; movl %edi, %ecx ; movq 0(%rdx), %rax -; movq %rax, %rdx -; addq %rdx, %rcx, %rdx +; lea 0(%rax,%rcx,1), %rcx ; cmpl %r11d, %edi -; cmovnbq %rax, %rdx, %rdx -; movq %rsi, 0(%rdx) +; cmovnbq %rax, %rcx, %rcx +; movq %rsi, 0(%rcx) ; movq %rbp, %rsp ; popq %rbp ; ret @@ -44,18 +43,17 @@ block0(v0: i32, v1: r64, v2: i64): ; block1: ; offset 0x4 ; movl 8(%rdx), %r11d ; cmpl %r11d, %edi -; jae 0x2b +; jae 0x28 ; block2: ; offset 0x11 ; movl %edi, %ecx ; movq (%rdx), %rax -; movq %rax, %rdx -; addq %rcx, %rdx +; addq %rax, %rcx ; cmpl %r11d, %edi -; cmovaeq %rax, %rdx -; movq %rsi, (%rdx) +; cmovaeq %rax, %rcx +; movq %rsi, (%rcx) ; movq %rbp, %rsp ; popq %rbp ; retq -; block3: ; offset 0x2b +; block3: ; offset 0x28 ; ud2 ; trap: table_oob diff --git a/cranelift/filetests/filetests/isa/x64/uextend-elision.clif b/cranelift/filetests/filetests/isa/x64/uextend-elision.clif index faf3a6763b67..d4e306f50f95 100644 --- a/cranelift/filetests/filetests/isa/x64/uextend-elision.clif +++ b/cranelift/filetests/filetests/isa/x64/uextend-elision.clif @@ -12,8 +12,7 @@ block0(v0: i32, v1: i32): ; pushq %rbp ; movq %rsp, %rbp ; block0: -; movq %rdi, %rax -; addl %eax, %esi, %eax +; lea 0(%rdi,%rsi,1), %eax ; movq %rbp, %rsp ; popq %rbp ; ret @@ -23,8 +22,7 @@ block0(v0: i32, v1: i32): ; pushq %rbp ; movq %rsp, %rbp ; block1: ; offset 0x4 -; movq %rdi, %rax -; addl %esi, %eax +; leal (%rdi, %rsi), %eax ; movq %rbp, %rsp ; popq %rbp ; retq diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0_guard_no_spectre_i32_access_0xffff0000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0_guard_no_spectre_i32_access_0xffff0000_offset.wat index b53718fbbb50..3010a8185f41 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0_guard_no_spectre_i32_access_0xffff0000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0_guard_no_spectre_i32_access_0xffff0000_offset.wat @@ -45,17 +45,17 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movl %edi, %r11d -;; movq %r11, %rdi -;; addq %rdi, const(1), %rdi +;; movl %edi, %r8d +;; movq %r8, %r11 +;; addq %r11, const(0), %r11 ;; jnb ; ud2 heap_oob ; -;; movq 8(%rdx), %rax -;; cmpq %rax, %rdi +;; movq 8(%rdx), %rdi +;; cmpq %rdi, %r11 ;; jnbe label3; j label1 ;; block1: -;; movq 0(%rdx), %rdi -;; addq %rdi, const(0), %rdi -;; movl %esi, 0(%rdi,%r11,1) +;; addq %r8, 0(%rdx), %r8 +;; movl $-65536, %eax +;; movl %esi, 0(%r8,%rax,1) ;; jmp label2 ;; block2: ;; movq %rbp, %rsp @@ -70,17 +70,17 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movl %edi, %r11d -;; movq %r11, %rdi -;; addq %rdi, const(1), %rdi +;; movl %edi, %r8d +;; movq %r8, %r11 +;; addq %r11, const(0), %r11 ;; jnb ; ud2 heap_oob ; -;; movq 8(%rsi), %rax -;; cmpq %rax, %rdi +;; movq 8(%rsi), %rdi +;; cmpq %rdi, %r11 ;; jnbe label3; j label1 ;; block1: -;; movq 0(%rsi), %rsi -;; addq %rsi, const(0), %rsi -;; movl 0(%rsi,%r11,1), %eax +;; addq %r8, 0(%rsi), %r8 +;; movl $-65536, %edi +;; movl 0(%r8,%rdi,1), %eax ;; jmp label2 ;; block2: ;; movq %rbp, %rsp diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0_guard_no_spectre_i8_access_0xffff0000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0_guard_no_spectre_i8_access_0xffff0000_offset.wat index b8c74b7a27be..5c3c1ac1e295 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0_guard_no_spectre_i8_access_0xffff0000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0_guard_no_spectre_i8_access_0xffff0000_offset.wat @@ -45,17 +45,17 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movl %edi, %r11d -;; movq %r11, %rdi -;; addq %rdi, const(1), %rdi +;; movl %edi, %r8d +;; movq %r8, %r11 +;; addq %r11, const(0), %r11 ;; jnb ; ud2 heap_oob ; -;; movq 8(%rdx), %rax -;; cmpq %rax, %rdi +;; movq 8(%rdx), %rdi +;; cmpq %rdi, %r11 ;; jnbe label3; j label1 ;; block1: -;; movq 0(%rdx), %rdi -;; addq %rdi, const(0), %rdi -;; movb %sil, 0(%rdi,%r11,1) +;; addq %r8, 0(%rdx), %r8 +;; movl $-65536, %eax +;; movb %sil, 0(%r8,%rax,1) ;; jmp label2 ;; block2: ;; movq %rbp, %rsp @@ -70,17 +70,17 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movl %edi, %r11d -;; movq %r11, %rdi -;; addq %rdi, const(1), %rdi +;; movl %edi, %r8d +;; movq %r8, %r11 +;; addq %r11, const(0), %r11 ;; jnb ; ud2 heap_oob ; -;; movq 8(%rsi), %rax -;; cmpq %rax, %rdi +;; movq 8(%rsi), %rdi +;; cmpq %rdi, %r11 ;; jnbe label3; j label1 ;; block1: -;; movq 0(%rsi), %rsi -;; addq %rsi, const(0), %rsi -;; movzbq 0(%rsi,%r11,1), %rax +;; addq %r8, 0(%rsi), %r8 +;; movl $-65536, %edi +;; movzbq 0(%r8,%rdi,1), %rax ;; jmp label2 ;; block2: ;; movq %rbp, %rsp diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0_guard_yes_spectre_i32_access_0x1000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0_guard_yes_spectre_i32_access_0x1000_offset.wat index 8bc688f26927..3c42b63e6be1 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0_guard_yes_spectre_i32_access_0x1000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0_guard_yes_spectre_i32_access_0x1000_offset.wat @@ -48,13 +48,12 @@ ;; movl %edi, %edi ;; movabsq $-4100, %rax ;; addq %rax, 8(%rdx), %rax -;; movq %rdi, %r11 -;; addq %r11, 0(%rdx), %r11 -;; addq %r11, $4096, %r11 -;; xorq %rcx, %rcx, %rcx +;; movq 0(%rdx), %rcx +;; lea 4096(%rcx,%rdi,1), %rcx +;; xorq %rdx, %rdx, %rdx ;; cmpq %rax, %rdi -;; cmovnbeq %rcx, %r11, %r11 -;; movl %esi, 0(%r11) +;; cmovnbeq %rdx, %rcx, %rcx +;; movl %esi, 0(%rcx) ;; jmp label1 ;; block1: ;; movq %rbp, %rsp @@ -70,15 +69,14 @@ ;; movl %edi, %edi ;; movabsq $-4100, %rax ;; addq %rax, 8(%rsi), %rax -;; movq %rdi, %r11 -;; addq %r11, 0(%rsi), %r11 -;; addq %r11, $4096, %r11 -;; xorq %rsi, %rsi, %rsi +;; movq 0(%rsi), %rcx +;; lea 4096(%rcx,%rdi,1), %rsi +;; xorq %rcx, %rcx, %rcx ;; cmpq %rax, %rdi -;; cmovnbeq %rsi, %r11, %r11 -;; movl 0(%r11), %eax +;; cmovnbeq %rcx, %rsi, %rsi +;; movl 0(%rsi), %eax ;; jmp label1 ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0_guard_yes_spectre_i32_access_0xffff0000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0_guard_yes_spectre_i32_access_0xffff0000_offset.wat index f4f79fff1cc5..412b47919310 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0_guard_yes_spectre_i32_access_0xffff0000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0_guard_yes_spectre_i32_access_0xffff0000_offset.wat @@ -45,17 +45,18 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movl %edi, %r8d -;; movq %r8, %rdi -;; addq %rdi, const(1), %rdi +;; movl %edi, %r11d +;; movq %r11, %rax +;; addq %rax, const(0), %rax ;; jnb ; ud2 heap_oob ; -;; movq 8(%rdx), %rax -;; addq %r8, 0(%rdx), %r8 -;; addq %r8, const(0), %r8 -;; xorq %rcx, %rcx, %rcx -;; cmpq %rax, %rdi -;; cmovnbeq %rcx, %r8, %r8 -;; movl %esi, 0(%r8) +;; movq 8(%rdx), %rcx +;; addq %r11, 0(%rdx), %r11 +;; movl $-65536, %edx +;; lea 0(%r11,%rdx,1), %rdx +;; xorq %r8, %r8, %r8 +;; cmpq %rcx, %rax +;; cmovnbeq %r8, %rdx, %rdx +;; movl %esi, 0(%rdx) ;; jmp label1 ;; block1: ;; movq %rbp, %rsp @@ -68,19 +69,20 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movl %edi, %r8d -;; movq %r8, %rdi -;; addq %rdi, const(1), %rdi +;; movl %edi, %r11d +;; movq %r11, %rax +;; addq %rax, const(0), %rax ;; jnb ; ud2 heap_oob ; -;; movq 8(%rsi), %rax -;; addq %r8, 0(%rsi), %r8 -;; addq %r8, const(0), %r8 -;; xorq %rcx, %rcx, %rcx -;; cmpq %rax, %rdi -;; cmovnbeq %rcx, %r8, %r8 -;; movl 0(%r8), %eax +;; movq 8(%rsi), %rcx +;; addq %r11, 0(%rsi), %r11 +;; movl $-65536, %edx +;; lea 0(%r11,%rdx,1), %rdx +;; xorq %r8, %r8, %r8 +;; cmpq %rcx, %rax +;; cmovnbeq %r8, %rdx, %rdx +;; movl 0(%rdx), %eax ;; jmp label1 ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0_guard_yes_spectre_i8_access_0x1000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0_guard_yes_spectre_i8_access_0x1000_offset.wat index f10feb9cb83d..69169611ba93 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0_guard_yes_spectre_i8_access_0x1000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0_guard_yes_spectre_i8_access_0x1000_offset.wat @@ -48,13 +48,12 @@ ;; movl %edi, %edi ;; movabsq $-4097, %rax ;; addq %rax, 8(%rdx), %rax -;; movq %rdi, %r11 -;; addq %r11, 0(%rdx), %r11 -;; addq %r11, $4096, %r11 -;; xorq %rcx, %rcx, %rcx +;; movq 0(%rdx), %rcx +;; lea 4096(%rcx,%rdi,1), %rcx +;; xorq %rdx, %rdx, %rdx ;; cmpq %rax, %rdi -;; cmovnbeq %rcx, %r11, %r11 -;; movb %sil, 0(%r11) +;; cmovnbeq %rdx, %rcx, %rcx +;; movb %sil, 0(%rcx) ;; jmp label1 ;; block1: ;; movq %rbp, %rsp @@ -70,15 +69,14 @@ ;; movl %edi, %edi ;; movabsq $-4097, %rax ;; addq %rax, 8(%rsi), %rax -;; movq %rdi, %r11 -;; addq %r11, 0(%rsi), %r11 -;; addq %r11, $4096, %r11 -;; xorq %rsi, %rsi, %rsi +;; movq 0(%rsi), %rcx +;; lea 4096(%rcx,%rdi,1), %rsi +;; xorq %rcx, %rcx, %rcx ;; cmpq %rax, %rdi -;; cmovnbeq %rsi, %r11, %r11 -;; movzbq 0(%r11), %rax +;; cmovnbeq %rcx, %rsi, %rsi +;; movzbq 0(%rsi), %rax ;; jmp label1 ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0_guard_yes_spectre_i8_access_0xffff0000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0_guard_yes_spectre_i8_access_0xffff0000_offset.wat index c68204b5d298..a4adf2663516 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0_guard_yes_spectre_i8_access_0xffff0000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0_guard_yes_spectre_i8_access_0xffff0000_offset.wat @@ -45,17 +45,18 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movl %edi, %r8d -;; movq %r8, %rdi -;; addq %rdi, const(1), %rdi +;; movl %edi, %r11d +;; movq %r11, %rax +;; addq %rax, const(0), %rax ;; jnb ; ud2 heap_oob ; -;; movq 8(%rdx), %rax -;; addq %r8, 0(%rdx), %r8 -;; addq %r8, const(0), %r8 -;; xorq %rcx, %rcx, %rcx -;; cmpq %rax, %rdi -;; cmovnbeq %rcx, %r8, %r8 -;; movb %sil, 0(%r8) +;; movq 8(%rdx), %rcx +;; addq %r11, 0(%rdx), %r11 +;; movl $-65536, %edx +;; lea 0(%r11,%rdx,1), %rdx +;; xorq %r8, %r8, %r8 +;; cmpq %rcx, %rax +;; cmovnbeq %r8, %rdx, %rdx +;; movb %sil, 0(%rdx) ;; jmp label1 ;; block1: ;; movq %rbp, %rsp @@ -68,19 +69,20 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movl %edi, %r8d -;; movq %r8, %rdi -;; addq %rdi, const(1), %rdi +;; movl %edi, %r11d +;; movq %r11, %rax +;; addq %rax, const(0), %rax ;; jnb ; ud2 heap_oob ; -;; movq 8(%rsi), %rax -;; addq %r8, 0(%rsi), %r8 -;; addq %r8, const(0), %r8 -;; xorq %rcx, %rcx, %rcx -;; cmpq %rax, %rdi -;; cmovnbeq %rcx, %r8, %r8 -;; movzbq 0(%r8), %rax +;; movq 8(%rsi), %rcx +;; addq %r11, 0(%rsi), %r11 +;; movl $-65536, %edx +;; lea 0(%r11,%rdx,1), %rdx +;; xorq %r8, %r8, %r8 +;; cmpq %rcx, %rax +;; cmovnbeq %r8, %rdx, %rdx +;; movzbq 0(%rdx), %rax ;; jmp label1 ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0xffffffff_guard_no_spectre_i32_access_0xffff0000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0xffffffff_guard_no_spectre_i32_access_0xffff0000_offset.wat index 41697bf726ed..2b8019e8501c 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0xffffffff_guard_no_spectre_i32_access_0xffff0000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0xffffffff_guard_no_spectre_i32_access_0xffff0000_offset.wat @@ -45,17 +45,17 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movl %edi, %r11d -;; movq %r11, %rdi -;; addq %rdi, const(1), %rdi +;; movl %edi, %r8d +;; movq %r8, %r11 +;; addq %r11, const(0), %r11 ;; jnb ; ud2 heap_oob ; -;; movq 8(%rdx), %rax -;; cmpq %rax, %rdi +;; movq 8(%rdx), %rdi +;; cmpq %rdi, %r11 ;; jnbe label3; j label1 ;; block1: -;; movq 0(%rdx), %rdi -;; addq %rdi, const(0), %rdi -;; movl %esi, 0(%rdi,%r11,1) +;; addq %r8, 0(%rdx), %r8 +;; movl $-65536, %eax +;; movl %esi, 0(%r8,%rax,1) ;; jmp label2 ;; block2: ;; movq %rbp, %rsp @@ -70,17 +70,17 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movl %edi, %r11d -;; movq %r11, %rdi -;; addq %rdi, const(1), %rdi +;; movl %edi, %r8d +;; movq %r8, %r11 +;; addq %r11, const(0), %r11 ;; jnb ; ud2 heap_oob ; -;; movq 8(%rsi), %rax -;; cmpq %rax, %rdi +;; movq 8(%rsi), %rdi +;; cmpq %rdi, %r11 ;; jnbe label3; j label1 ;; block1: -;; movq 0(%rsi), %rsi -;; addq %rsi, const(0), %rsi -;; movl 0(%rsi,%r11,1), %eax +;; addq %r8, 0(%rsi), %r8 +;; movl $-65536, %edi +;; movl 0(%r8,%rdi,1), %eax ;; jmp label2 ;; block2: ;; movq %rbp, %rsp diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0xffffffff_guard_no_spectre_i8_access_0xffff0000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0xffffffff_guard_no_spectre_i8_access_0xffff0000_offset.wat index cc2c6f792c31..29ff03b82058 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0xffffffff_guard_no_spectre_i8_access_0xffff0000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0xffffffff_guard_no_spectre_i8_access_0xffff0000_offset.wat @@ -45,17 +45,17 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movl %edi, %r11d -;; movq %r11, %rdi -;; addq %rdi, const(1), %rdi +;; movl %edi, %r8d +;; movq %r8, %r11 +;; addq %r11, const(0), %r11 ;; jnb ; ud2 heap_oob ; -;; movq 8(%rdx), %rax -;; cmpq %rax, %rdi +;; movq 8(%rdx), %rdi +;; cmpq %rdi, %r11 ;; jnbe label3; j label1 ;; block1: -;; movq 0(%rdx), %rdi -;; addq %rdi, const(0), %rdi -;; movb %sil, 0(%rdi,%r11,1) +;; addq %r8, 0(%rdx), %r8 +;; movl $-65536, %eax +;; movb %sil, 0(%r8,%rax,1) ;; jmp label2 ;; block2: ;; movq %rbp, %rsp @@ -70,17 +70,17 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movl %edi, %r11d -;; movq %r11, %rdi -;; addq %rdi, const(1), %rdi +;; movl %edi, %r8d +;; movq %r8, %r11 +;; addq %r11, const(0), %r11 ;; jnb ; ud2 heap_oob ; -;; movq 8(%rsi), %rax -;; cmpq %rax, %rdi +;; movq 8(%rsi), %rdi +;; cmpq %rdi, %r11 ;; jnbe label3; j label1 ;; block1: -;; movq 0(%rsi), %rsi -;; addq %rsi, const(0), %rsi -;; movzbq 0(%rsi,%r11,1), %rax +;; addq %r8, 0(%rsi), %r8 +;; movl $-65536, %edi +;; movzbq 0(%r8,%rdi,1), %rax ;; jmp label2 ;; block2: ;; movq %rbp, %rsp diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0xffffffff_guard_yes_spectre_i32_access_0x1000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0xffffffff_guard_yes_spectre_i32_access_0x1000_offset.wat index e9cc1c103323..9b8f9e4b37b6 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0xffffffff_guard_yes_spectre_i32_access_0x1000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0xffffffff_guard_yes_spectre_i32_access_0x1000_offset.wat @@ -48,13 +48,12 @@ ;; movl %edi, %edi ;; movabsq $-4100, %rax ;; addq %rax, 8(%rdx), %rax -;; movq %rdi, %r11 -;; addq %r11, 0(%rdx), %r11 -;; addq %r11, $4096, %r11 -;; xorq %rcx, %rcx, %rcx +;; movq 0(%rdx), %rcx +;; lea 4096(%rcx,%rdi,1), %rcx +;; xorq %rdx, %rdx, %rdx ;; cmpq %rax, %rdi -;; cmovnbeq %rcx, %r11, %r11 -;; movl %esi, 0(%r11) +;; cmovnbeq %rdx, %rcx, %rcx +;; movl %esi, 0(%rcx) ;; jmp label1 ;; block1: ;; movq %rbp, %rsp @@ -70,15 +69,14 @@ ;; movl %edi, %edi ;; movabsq $-4100, %rax ;; addq %rax, 8(%rsi), %rax -;; movq %rdi, %r11 -;; addq %r11, 0(%rsi), %r11 -;; addq %r11, $4096, %r11 -;; xorq %rsi, %rsi, %rsi +;; movq 0(%rsi), %rcx +;; lea 4096(%rcx,%rdi,1), %rsi +;; xorq %rcx, %rcx, %rcx ;; cmpq %rax, %rdi -;; cmovnbeq %rsi, %r11, %r11 -;; movl 0(%r11), %eax +;; cmovnbeq %rcx, %rsi, %rsi +;; movl 0(%rsi), %eax ;; jmp label1 ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0xffffffff_guard_yes_spectre_i32_access_0xffff0000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0xffffffff_guard_yes_spectre_i32_access_0xffff0000_offset.wat index f4ffb0d0f1f8..37401d35287d 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0xffffffff_guard_yes_spectre_i32_access_0xffff0000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0xffffffff_guard_yes_spectre_i32_access_0xffff0000_offset.wat @@ -45,17 +45,18 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movl %edi, %r8d -;; movq %r8, %rdi -;; addq %rdi, const(1), %rdi +;; movl %edi, %r11d +;; movq %r11, %rax +;; addq %rax, const(0), %rax ;; jnb ; ud2 heap_oob ; -;; movq 8(%rdx), %rax -;; addq %r8, 0(%rdx), %r8 -;; addq %r8, const(0), %r8 -;; xorq %rcx, %rcx, %rcx -;; cmpq %rax, %rdi -;; cmovnbeq %rcx, %r8, %r8 -;; movl %esi, 0(%r8) +;; movq 8(%rdx), %rcx +;; addq %r11, 0(%rdx), %r11 +;; movl $-65536, %edx +;; lea 0(%r11,%rdx,1), %rdx +;; xorq %r8, %r8, %r8 +;; cmpq %rcx, %rax +;; cmovnbeq %r8, %rdx, %rdx +;; movl %esi, 0(%rdx) ;; jmp label1 ;; block1: ;; movq %rbp, %rsp @@ -68,19 +69,20 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movl %edi, %r8d -;; movq %r8, %rdi -;; addq %rdi, const(1), %rdi +;; movl %edi, %r11d +;; movq %r11, %rax +;; addq %rax, const(0), %rax ;; jnb ; ud2 heap_oob ; -;; movq 8(%rsi), %rax -;; addq %r8, 0(%rsi), %r8 -;; addq %r8, const(0), %r8 -;; xorq %rcx, %rcx, %rcx -;; cmpq %rax, %rdi -;; cmovnbeq %rcx, %r8, %r8 -;; movl 0(%r8), %eax +;; movq 8(%rsi), %rcx +;; addq %r11, 0(%rsi), %r11 +;; movl $-65536, %edx +;; lea 0(%r11,%rdx,1), %rdx +;; xorq %r8, %r8, %r8 +;; cmpq %rcx, %rax +;; cmovnbeq %r8, %rdx, %rdx +;; movl 0(%rdx), %eax ;; jmp label1 ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0xffffffff_guard_yes_spectre_i8_access_0x1000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0xffffffff_guard_yes_spectre_i8_access_0x1000_offset.wat index a5d43ce9c316..99113750d30b 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0xffffffff_guard_yes_spectre_i8_access_0x1000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0xffffffff_guard_yes_spectre_i8_access_0x1000_offset.wat @@ -48,13 +48,12 @@ ;; movl %edi, %edi ;; movabsq $-4097, %rax ;; addq %rax, 8(%rdx), %rax -;; movq %rdi, %r11 -;; addq %r11, 0(%rdx), %r11 -;; addq %r11, $4096, %r11 -;; xorq %rcx, %rcx, %rcx +;; movq 0(%rdx), %rcx +;; lea 4096(%rcx,%rdi,1), %rcx +;; xorq %rdx, %rdx, %rdx ;; cmpq %rax, %rdi -;; cmovnbeq %rcx, %r11, %r11 -;; movb %sil, 0(%r11) +;; cmovnbeq %rdx, %rcx, %rcx +;; movb %sil, 0(%rcx) ;; jmp label1 ;; block1: ;; movq %rbp, %rsp @@ -70,15 +69,14 @@ ;; movl %edi, %edi ;; movabsq $-4097, %rax ;; addq %rax, 8(%rsi), %rax -;; movq %rdi, %r11 -;; addq %r11, 0(%rsi), %r11 -;; addq %r11, $4096, %r11 -;; xorq %rsi, %rsi, %rsi +;; movq 0(%rsi), %rcx +;; lea 4096(%rcx,%rdi,1), %rsi +;; xorq %rcx, %rcx, %rcx ;; cmpq %rax, %rdi -;; cmovnbeq %rsi, %r11, %r11 -;; movzbq 0(%r11), %rax +;; cmovnbeq %rcx, %rsi, %rsi +;; movzbq 0(%rsi), %rax ;; jmp label1 ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0xffffffff_guard_yes_spectre_i8_access_0xffff0000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0xffffffff_guard_yes_spectre_i8_access_0xffff0000_offset.wat index a67a3975ab7b..05ca250ad201 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0xffffffff_guard_yes_spectre_i8_access_0xffff0000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i32_index_0xffffffff_guard_yes_spectre_i8_access_0xffff0000_offset.wat @@ -45,17 +45,18 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movl %edi, %r8d -;; movq %r8, %rdi -;; addq %rdi, const(1), %rdi +;; movl %edi, %r11d +;; movq %r11, %rax +;; addq %rax, const(0), %rax ;; jnb ; ud2 heap_oob ; -;; movq 8(%rdx), %rax -;; addq %r8, 0(%rdx), %r8 -;; addq %r8, const(0), %r8 -;; xorq %rcx, %rcx, %rcx -;; cmpq %rax, %rdi -;; cmovnbeq %rcx, %r8, %r8 -;; movb %sil, 0(%r8) +;; movq 8(%rdx), %rcx +;; addq %r11, 0(%rdx), %r11 +;; movl $-65536, %edx +;; lea 0(%r11,%rdx,1), %rdx +;; xorq %r8, %r8, %r8 +;; cmpq %rcx, %rax +;; cmovnbeq %r8, %rdx, %rdx +;; movb %sil, 0(%rdx) ;; jmp label1 ;; block1: ;; movq %rbp, %rsp @@ -68,19 +69,20 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movl %edi, %r8d -;; movq %r8, %rdi -;; addq %rdi, const(1), %rdi +;; movl %edi, %r11d +;; movq %r11, %rax +;; addq %rax, const(0), %rax ;; jnb ; ud2 heap_oob ; -;; movq 8(%rsi), %rax -;; addq %r8, 0(%rsi), %r8 -;; addq %r8, const(0), %r8 -;; xorq %rcx, %rcx, %rcx -;; cmpq %rax, %rdi -;; cmovnbeq %rcx, %r8, %r8 -;; movzbq 0(%r8), %rax +;; movq 8(%rsi), %rcx +;; addq %r11, 0(%rsi), %r11 +;; movl $-65536, %edx +;; lea 0(%r11,%rdx,1), %rdx +;; xorq %r8, %r8, %r8 +;; cmpq %rcx, %rax +;; cmovnbeq %r8, %rdx, %rdx +;; movzbq 0(%rdx), %rax ;; jmp label1 ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0_guard_no_spectre_i32_access_0xffff0000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0_guard_no_spectre_i32_access_0xffff0000_offset.wat index 583769bad8d4..277b469a816e 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0_guard_no_spectre_i32_access_0xffff0000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0_guard_no_spectre_i32_access_0xffff0000_offset.wat @@ -46,15 +46,15 @@ ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: ;; movq %rdi, %r10 -;; addq %r10, const(1), %r10 +;; addq %r10, const(0), %r10 ;; jnb ; ud2 heap_oob ; ;; movq 8(%rdx), %r11 ;; cmpq %r11, %r10 ;; jnbe label3; j label1 ;; block1: -;; movq 0(%rdx), %rax -;; addq %rax, const(0), %rax -;; movl %esi, 0(%rax,%rdi,1) +;; addq %rdi, 0(%rdx), %rdi +;; movl $-65536, %eax +;; movl %esi, 0(%rdi,%rax,1) ;; jmp label2 ;; block2: ;; movq %rbp, %rsp @@ -70,15 +70,15 @@ ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: ;; movq %rdi, %r10 -;; addq %r10, const(1), %r10 +;; addq %r10, const(0), %r10 ;; jnb ; ud2 heap_oob ; ;; movq 8(%rsi), %r11 ;; cmpq %r11, %r10 ;; jnbe label3; j label1 ;; block1: -;; movq 0(%rsi), %r11 -;; addq %r11, const(0), %r11 -;; movl 0(%r11,%rdi,1), %eax +;; addq %rdi, 0(%rsi), %rdi +;; movl $-65536, %esi +;; movl 0(%rdi,%rsi,1), %eax ;; jmp label2 ;; block2: ;; movq %rbp, %rsp diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0_guard_no_spectre_i8_access_0xffff0000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0_guard_no_spectre_i8_access_0xffff0000_offset.wat index fb540d9fb469..a790538c98d7 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0_guard_no_spectre_i8_access_0xffff0000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0_guard_no_spectre_i8_access_0xffff0000_offset.wat @@ -46,15 +46,15 @@ ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: ;; movq %rdi, %r10 -;; addq %r10, const(1), %r10 +;; addq %r10, const(0), %r10 ;; jnb ; ud2 heap_oob ; ;; movq 8(%rdx), %r11 ;; cmpq %r11, %r10 ;; jnbe label3; j label1 ;; block1: -;; movq 0(%rdx), %rax -;; addq %rax, const(0), %rax -;; movb %sil, 0(%rax,%rdi,1) +;; addq %rdi, 0(%rdx), %rdi +;; movl $-65536, %eax +;; movb %sil, 0(%rdi,%rax,1) ;; jmp label2 ;; block2: ;; movq %rbp, %rsp @@ -70,15 +70,15 @@ ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: ;; movq %rdi, %r10 -;; addq %r10, const(1), %r10 +;; addq %r10, const(0), %r10 ;; jnb ; ud2 heap_oob ; ;; movq 8(%rsi), %r11 ;; cmpq %r11, %r10 ;; jnbe label3; j label1 ;; block1: -;; movq 0(%rsi), %r11 -;; addq %r11, const(0), %r11 -;; movzbq 0(%r11,%rdi,1), %rax +;; addq %rdi, 0(%rsi), %rdi +;; movl $-65536, %esi +;; movzbq 0(%rdi,%rsi,1), %rax ;; jmp label2 ;; block2: ;; movq %rbp, %rsp diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0_guard_yes_spectre_i32_access_0x1000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0_guard_yes_spectre_i32_access_0x1000_offset.wat index 6a2274169103..ca9a9ac97e9d 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0_guard_yes_spectre_i32_access_0x1000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0_guard_yes_spectre_i32_access_0x1000_offset.wat @@ -47,13 +47,12 @@ ;; block0: ;; movabsq $-4100, %rax ;; addq %rax, 8(%rdx), %rax -;; movq %rdi, %r10 -;; addq %r10, 0(%rdx), %r10 -;; addq %r10, $4096, %r10 -;; xorq %r11, %r11, %r11 +;; movq 0(%rdx), %rcx +;; lea 4096(%rcx,%rdi,1), %r11 +;; xorq %rcx, %rcx, %rcx ;; cmpq %rax, %rdi -;; cmovnbeq %r11, %r10, %r10 -;; movl %esi, 0(%r10) +;; cmovnbeq %rcx, %r11, %r11 +;; movl %esi, 0(%r11) ;; jmp label1 ;; block1: ;; movq %rbp, %rsp @@ -66,18 +65,16 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movq %rsi, %rax -;; movabsq $-4100, %rsi -;; addq %rsi, 8(%rax), %rsi -;; movq %rdi, %r10 -;; addq %r10, 0(%rax), %r10 -;; addq %r10, $4096, %r10 -;; xorq %r11, %r11, %r11 -;; cmpq %rsi, %rdi -;; cmovnbeq %r11, %r10, %r10 -;; movl 0(%r10), %eax +;; movabsq $-4100, %rcx +;; addq %rcx, 8(%rsi), %rcx +;; movq 0(%rsi), %rsi +;; lea 4096(%rsi,%rdi,1), %r11 +;; xorq %rax, %rax, %rax +;; cmpq %rcx, %rdi +;; cmovnbeq %rax, %r11, %r11 +;; movl 0(%r11), %eax ;; jmp label1 ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0_guard_yes_spectre_i32_access_0xffff0000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0_guard_yes_spectre_i32_access_0xffff0000_offset.wat index 0a1c3e7e4e42..369f5146af23 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0_guard_yes_spectre_i32_access_0xffff0000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0_guard_yes_spectre_i32_access_0xffff0000_offset.wat @@ -45,17 +45,18 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movq %rdi, %rcx -;; addq %rcx, const(1), %rcx +;; movq %rdi, %r8 +;; addq %r8, const(0), %r8 ;; jnb ; ud2 heap_oob ; ;; movq 8(%rdx), %rax -;; movq %rdi, %r11 -;; addq %r11, 0(%rdx), %r11 -;; addq %r11, const(0), %r11 +;; movq %rdi, %rcx +;; addq %rcx, 0(%rdx), %rcx +;; movl $-65536, %edi +;; lea 0(%rcx,%rdi,1), %rcx ;; xorq %rdi, %rdi, %rdi -;; cmpq %rax, %rcx -;; cmovnbeq %rdi, %r11, %r11 -;; movl %esi, 0(%r11) +;; cmpq %rax, %r8 +;; cmovnbeq %rdi, %rcx, %rcx +;; movl %esi, 0(%rcx) ;; jmp label1 ;; block1: ;; movq %rbp, %rsp @@ -68,19 +69,20 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movq %rdi, %rcx -;; addq %rcx, const(1), %rcx +;; movq %rdi, %rdx +;; addq %rdx, const(0), %rdx ;; jnb ; ud2 heap_oob ; ;; movq 8(%rsi), %rax -;; movq %rdi, %r11 -;; addq %r11, 0(%rsi), %r11 -;; addq %r11, const(0), %r11 -;; xorq %rsi, %rsi, %rsi -;; cmpq %rax, %rcx -;; cmovnbeq %rsi, %r11, %r11 -;; movl 0(%r11), %eax +;; movq %rdi, %rcx +;; addq %rcx, 0(%rsi), %rcx +;; movl $-65536, %edi +;; lea 0(%rcx,%rdi,1), %rcx +;; xorq %rdi, %rdi, %rdi +;; cmpq %rax, %rdx +;; cmovnbeq %rdi, %rcx, %rcx +;; movl 0(%rcx), %eax ;; jmp label1 ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0_guard_yes_spectre_i8_access_0x1000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0_guard_yes_spectre_i8_access_0x1000_offset.wat index 7e1de03d11dc..28522fed9b33 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0_guard_yes_spectre_i8_access_0x1000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0_guard_yes_spectre_i8_access_0x1000_offset.wat @@ -47,13 +47,12 @@ ;; block0: ;; movabsq $-4097, %rax ;; addq %rax, 8(%rdx), %rax -;; movq %rdi, %r10 -;; addq %r10, 0(%rdx), %r10 -;; addq %r10, $4096, %r10 -;; xorq %r11, %r11, %r11 +;; movq 0(%rdx), %rcx +;; lea 4096(%rcx,%rdi,1), %r11 +;; xorq %rcx, %rcx, %rcx ;; cmpq %rax, %rdi -;; cmovnbeq %r11, %r10, %r10 -;; movb %sil, 0(%r10) +;; cmovnbeq %rcx, %r11, %r11 +;; movb %sil, 0(%r11) ;; jmp label1 ;; block1: ;; movq %rbp, %rsp @@ -66,18 +65,16 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movq %rsi, %rax -;; movabsq $-4097, %rsi -;; addq %rsi, 8(%rax), %rsi -;; movq %rdi, %r10 -;; addq %r10, 0(%rax), %r10 -;; addq %r10, $4096, %r10 -;; xorq %r11, %r11, %r11 -;; cmpq %rsi, %rdi -;; cmovnbeq %r11, %r10, %r10 -;; movzbq 0(%r10), %rax +;; movabsq $-4097, %rcx +;; addq %rcx, 8(%rsi), %rcx +;; movq 0(%rsi), %rsi +;; lea 4096(%rsi,%rdi,1), %r11 +;; xorq %rax, %rax, %rax +;; cmpq %rcx, %rdi +;; cmovnbeq %rax, %r11, %r11 +;; movzbq 0(%r11), %rax ;; jmp label1 ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0_guard_yes_spectre_i8_access_0xffff0000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0_guard_yes_spectre_i8_access_0xffff0000_offset.wat index b29744517616..e3aa79c864d0 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0_guard_yes_spectre_i8_access_0xffff0000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0_guard_yes_spectre_i8_access_0xffff0000_offset.wat @@ -45,17 +45,18 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movq %rdi, %rcx -;; addq %rcx, const(1), %rcx +;; movq %rdi, %r8 +;; addq %r8, const(0), %r8 ;; jnb ; ud2 heap_oob ; ;; movq 8(%rdx), %rax -;; movq %rdi, %r11 -;; addq %r11, 0(%rdx), %r11 -;; addq %r11, const(0), %r11 +;; movq %rdi, %rcx +;; addq %rcx, 0(%rdx), %rcx +;; movl $-65536, %edi +;; lea 0(%rcx,%rdi,1), %rcx ;; xorq %rdi, %rdi, %rdi -;; cmpq %rax, %rcx -;; cmovnbeq %rdi, %r11, %r11 -;; movb %sil, 0(%r11) +;; cmpq %rax, %r8 +;; cmovnbeq %rdi, %rcx, %rcx +;; movb %sil, 0(%rcx) ;; jmp label1 ;; block1: ;; movq %rbp, %rsp @@ -68,19 +69,20 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movq %rdi, %rcx -;; addq %rcx, const(1), %rcx +;; movq %rdi, %rdx +;; addq %rdx, const(0), %rdx ;; jnb ; ud2 heap_oob ; ;; movq 8(%rsi), %rax -;; movq %rdi, %r11 -;; addq %r11, 0(%rsi), %r11 -;; addq %r11, const(0), %r11 -;; xorq %rsi, %rsi, %rsi -;; cmpq %rax, %rcx -;; cmovnbeq %rsi, %r11, %r11 -;; movzbq 0(%r11), %rax +;; movq %rdi, %rcx +;; addq %rcx, 0(%rsi), %rcx +;; movl $-65536, %edi +;; lea 0(%rcx,%rdi,1), %rcx +;; xorq %rdi, %rdi, %rdi +;; cmpq %rax, %rdx +;; cmovnbeq %rdi, %rcx, %rcx +;; movzbq 0(%rcx), %rax ;; jmp label1 ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0xffffffff_guard_no_spectre_i32_access_0xffff0000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0xffffffff_guard_no_spectre_i32_access_0xffff0000_offset.wat index 3f4ec47b0172..95d01f56a062 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0xffffffff_guard_no_spectre_i32_access_0xffff0000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0xffffffff_guard_no_spectre_i32_access_0xffff0000_offset.wat @@ -46,15 +46,15 @@ ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: ;; movq %rdi, %r10 -;; addq %r10, const(1), %r10 +;; addq %r10, const(0), %r10 ;; jnb ; ud2 heap_oob ; ;; movq 8(%rdx), %r11 ;; cmpq %r11, %r10 ;; jnbe label3; j label1 ;; block1: -;; movq 0(%rdx), %rax -;; addq %rax, const(0), %rax -;; movl %esi, 0(%rax,%rdi,1) +;; addq %rdi, 0(%rdx), %rdi +;; movl $-65536, %eax +;; movl %esi, 0(%rdi,%rax,1) ;; jmp label2 ;; block2: ;; movq %rbp, %rsp @@ -70,15 +70,15 @@ ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: ;; movq %rdi, %r10 -;; addq %r10, const(1), %r10 +;; addq %r10, const(0), %r10 ;; jnb ; ud2 heap_oob ; ;; movq 8(%rsi), %r11 ;; cmpq %r11, %r10 ;; jnbe label3; j label1 ;; block1: -;; movq 0(%rsi), %r11 -;; addq %r11, const(0), %r11 -;; movl 0(%r11,%rdi,1), %eax +;; addq %rdi, 0(%rsi), %rdi +;; movl $-65536, %esi +;; movl 0(%rdi,%rsi,1), %eax ;; jmp label2 ;; block2: ;; movq %rbp, %rsp diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0xffffffff_guard_no_spectre_i8_access_0xffff0000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0xffffffff_guard_no_spectre_i8_access_0xffff0000_offset.wat index 9d20727290f0..ebc17c8894c0 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0xffffffff_guard_no_spectre_i8_access_0xffff0000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0xffffffff_guard_no_spectre_i8_access_0xffff0000_offset.wat @@ -46,15 +46,15 @@ ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: ;; movq %rdi, %r10 -;; addq %r10, const(1), %r10 +;; addq %r10, const(0), %r10 ;; jnb ; ud2 heap_oob ; ;; movq 8(%rdx), %r11 ;; cmpq %r11, %r10 ;; jnbe label3; j label1 ;; block1: -;; movq 0(%rdx), %rax -;; addq %rax, const(0), %rax -;; movb %sil, 0(%rax,%rdi,1) +;; addq %rdi, 0(%rdx), %rdi +;; movl $-65536, %eax +;; movb %sil, 0(%rdi,%rax,1) ;; jmp label2 ;; block2: ;; movq %rbp, %rsp @@ -70,15 +70,15 @@ ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: ;; movq %rdi, %r10 -;; addq %r10, const(1), %r10 +;; addq %r10, const(0), %r10 ;; jnb ; ud2 heap_oob ; ;; movq 8(%rsi), %r11 ;; cmpq %r11, %r10 ;; jnbe label3; j label1 ;; block1: -;; movq 0(%rsi), %r11 -;; addq %r11, const(0), %r11 -;; movzbq 0(%r11,%rdi,1), %rax +;; addq %rdi, 0(%rsi), %rdi +;; movl $-65536, %esi +;; movzbq 0(%rdi,%rsi,1), %rax ;; jmp label2 ;; block2: ;; movq %rbp, %rsp diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0xffffffff_guard_yes_spectre_i32_access_0x1000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0xffffffff_guard_yes_spectre_i32_access_0x1000_offset.wat index 7e7b96928778..99f3c89c12a0 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0xffffffff_guard_yes_spectre_i32_access_0x1000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0xffffffff_guard_yes_spectre_i32_access_0x1000_offset.wat @@ -47,13 +47,12 @@ ;; block0: ;; movabsq $-4100, %rax ;; addq %rax, 8(%rdx), %rax -;; movq %rdi, %r10 -;; addq %r10, 0(%rdx), %r10 -;; addq %r10, $4096, %r10 -;; xorq %r11, %r11, %r11 +;; movq 0(%rdx), %rcx +;; lea 4096(%rcx,%rdi,1), %r11 +;; xorq %rcx, %rcx, %rcx ;; cmpq %rax, %rdi -;; cmovnbeq %r11, %r10, %r10 -;; movl %esi, 0(%r10) +;; cmovnbeq %rcx, %r11, %r11 +;; movl %esi, 0(%r11) ;; jmp label1 ;; block1: ;; movq %rbp, %rsp @@ -66,18 +65,16 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movq %rsi, %rax -;; movabsq $-4100, %rsi -;; addq %rsi, 8(%rax), %rsi -;; movq %rdi, %r10 -;; addq %r10, 0(%rax), %r10 -;; addq %r10, $4096, %r10 -;; xorq %r11, %r11, %r11 -;; cmpq %rsi, %rdi -;; cmovnbeq %r11, %r10, %r10 -;; movl 0(%r10), %eax +;; movabsq $-4100, %rcx +;; addq %rcx, 8(%rsi), %rcx +;; movq 0(%rsi), %rsi +;; lea 4096(%rsi,%rdi,1), %r11 +;; xorq %rax, %rax, %rax +;; cmpq %rcx, %rdi +;; cmovnbeq %rax, %r11, %r11 +;; movl 0(%r11), %eax ;; jmp label1 ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0xffffffff_guard_yes_spectre_i32_access_0xffff0000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0xffffffff_guard_yes_spectre_i32_access_0xffff0000_offset.wat index 6368789603ec..95a7e4afff45 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0xffffffff_guard_yes_spectre_i32_access_0xffff0000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0xffffffff_guard_yes_spectre_i32_access_0xffff0000_offset.wat @@ -45,17 +45,18 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movq %rdi, %rcx -;; addq %rcx, const(1), %rcx +;; movq %rdi, %r8 +;; addq %r8, const(0), %r8 ;; jnb ; ud2 heap_oob ; ;; movq 8(%rdx), %rax -;; movq %rdi, %r11 -;; addq %r11, 0(%rdx), %r11 -;; addq %r11, const(0), %r11 +;; movq %rdi, %rcx +;; addq %rcx, 0(%rdx), %rcx +;; movl $-65536, %edi +;; lea 0(%rcx,%rdi,1), %rcx ;; xorq %rdi, %rdi, %rdi -;; cmpq %rax, %rcx -;; cmovnbeq %rdi, %r11, %r11 -;; movl %esi, 0(%r11) +;; cmpq %rax, %r8 +;; cmovnbeq %rdi, %rcx, %rcx +;; movl %esi, 0(%rcx) ;; jmp label1 ;; block1: ;; movq %rbp, %rsp @@ -68,19 +69,20 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movq %rdi, %rcx -;; addq %rcx, const(1), %rcx +;; movq %rdi, %rdx +;; addq %rdx, const(0), %rdx ;; jnb ; ud2 heap_oob ; ;; movq 8(%rsi), %rax -;; movq %rdi, %r11 -;; addq %r11, 0(%rsi), %r11 -;; addq %r11, const(0), %r11 -;; xorq %rsi, %rsi, %rsi -;; cmpq %rax, %rcx -;; cmovnbeq %rsi, %r11, %r11 -;; movl 0(%r11), %eax +;; movq %rdi, %rcx +;; addq %rcx, 0(%rsi), %rcx +;; movl $-65536, %edi +;; lea 0(%rcx,%rdi,1), %rcx +;; xorq %rdi, %rdi, %rdi +;; cmpq %rax, %rdx +;; cmovnbeq %rdi, %rcx, %rcx +;; movl 0(%rcx), %eax ;; jmp label1 ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0xffffffff_guard_yes_spectre_i8_access_0x1000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0xffffffff_guard_yes_spectre_i8_access_0x1000_offset.wat index ba1cf005500b..a0cffc83b564 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0xffffffff_guard_yes_spectre_i8_access_0x1000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0xffffffff_guard_yes_spectre_i8_access_0x1000_offset.wat @@ -47,13 +47,12 @@ ;; block0: ;; movabsq $-4097, %rax ;; addq %rax, 8(%rdx), %rax -;; movq %rdi, %r10 -;; addq %r10, 0(%rdx), %r10 -;; addq %r10, $4096, %r10 -;; xorq %r11, %r11, %r11 +;; movq 0(%rdx), %rcx +;; lea 4096(%rcx,%rdi,1), %r11 +;; xorq %rcx, %rcx, %rcx ;; cmpq %rax, %rdi -;; cmovnbeq %r11, %r10, %r10 -;; movb %sil, 0(%r10) +;; cmovnbeq %rcx, %r11, %r11 +;; movb %sil, 0(%r11) ;; jmp label1 ;; block1: ;; movq %rbp, %rsp @@ -66,18 +65,16 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movq %rsi, %rax -;; movabsq $-4097, %rsi -;; addq %rsi, 8(%rax), %rsi -;; movq %rdi, %r10 -;; addq %r10, 0(%rax), %r10 -;; addq %r10, $4096, %r10 -;; xorq %r11, %r11, %r11 -;; cmpq %rsi, %rdi -;; cmovnbeq %r11, %r10, %r10 -;; movzbq 0(%r10), %rax +;; movabsq $-4097, %rcx +;; addq %rcx, 8(%rsi), %rcx +;; movq 0(%rsi), %rsi +;; lea 4096(%rsi,%rdi,1), %r11 +;; xorq %rax, %rax, %rax +;; cmpq %rcx, %rdi +;; cmovnbeq %rax, %r11, %r11 +;; movzbq 0(%r11), %rax ;; jmp label1 ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0xffffffff_guard_yes_spectre_i8_access_0xffff0000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0xffffffff_guard_yes_spectre_i8_access_0xffff0000_offset.wat index 003dd6aa8598..20977fe0116a 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0xffffffff_guard_yes_spectre_i8_access_0xffff0000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_dynamic_kind_i64_index_0xffffffff_guard_yes_spectre_i8_access_0xffff0000_offset.wat @@ -45,17 +45,18 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movq %rdi, %rcx -;; addq %rcx, const(1), %rcx +;; movq %rdi, %r8 +;; addq %r8, const(0), %r8 ;; jnb ; ud2 heap_oob ; ;; movq 8(%rdx), %rax -;; movq %rdi, %r11 -;; addq %r11, 0(%rdx), %r11 -;; addq %r11, const(0), %r11 +;; movq %rdi, %rcx +;; addq %rcx, 0(%rdx), %rcx +;; movl $-65536, %edi +;; lea 0(%rcx,%rdi,1), %rcx ;; xorq %rdi, %rdi, %rdi -;; cmpq %rax, %rcx -;; cmovnbeq %rdi, %r11, %r11 -;; movb %sil, 0(%r11) +;; cmpq %rax, %r8 +;; cmovnbeq %rdi, %rcx, %rcx +;; movb %sil, 0(%rcx) ;; jmp label1 ;; block1: ;; movq %rbp, %rsp @@ -68,19 +69,20 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movq %rdi, %rcx -;; addq %rcx, const(1), %rcx +;; movq %rdi, %rdx +;; addq %rdx, const(0), %rdx ;; jnb ; ud2 heap_oob ; ;; movq 8(%rsi), %rax -;; movq %rdi, %r11 -;; addq %r11, 0(%rsi), %r11 -;; addq %r11, const(0), %r11 -;; xorq %rsi, %rsi, %rsi -;; cmpq %rax, %rcx -;; cmovnbeq %rsi, %r11, %r11 -;; movzbq 0(%r11), %rax +;; movq %rdi, %rcx +;; addq %rcx, 0(%rsi), %rcx +;; movl $-65536, %edi +;; lea 0(%rcx,%rdi,1), %rcx +;; xorq %rdi, %rdi, %rdi +;; cmpq %rax, %rdx +;; cmovnbeq %rdi, %rcx, %rcx +;; movzbq 0(%rcx), %rax ;; jmp label1 ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i32_index_0_guard_yes_spectre_i32_access_0x1000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i32_index_0_guard_yes_spectre_i32_access_0x1000_offset.wat index 2f434668a1e3..3a6c1b9dcbb7 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i32_index_0_guard_yes_spectre_i32_access_0x1000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i32_index_0_guard_yes_spectre_i32_access_0x1000_offset.wat @@ -44,13 +44,12 @@ ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: ;; movl %edi, %r11d -;; movq %r11, %r9 -;; addq %r9, 0(%rdx), %r9 -;; addq %r9, $4096, %r9 -;; xorq %r10, %r10, %r10 +;; movq 0(%rdx), %rdi +;; lea 4096(%rdi,%r11,1), %r10 +;; xorq %rdi, %rdi, %rdi ;; cmpq $268431356, %r11 -;; cmovnbeq %r10, %r9, %r9 -;; movl %esi, 0(%r9) +;; cmovnbeq %rdi, %r10, %r10 +;; movl %esi, 0(%r10) ;; jmp label1 ;; block1: ;; movq %rbp, %rsp @@ -64,15 +63,14 @@ ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: ;; movl %edi, %r11d -;; movq %r11, %r9 -;; addq %r9, 0(%rsi), %r9 -;; addq %r9, $4096, %r9 -;; xorq %r10, %r10, %r10 +;; movq 0(%rsi), %rsi +;; lea 4096(%rsi,%r11,1), %r10 +;; xorq %rsi, %rsi, %rsi ;; cmpq $268431356, %r11 -;; cmovnbeq %r10, %r9, %r9 -;; movl 0(%r9), %eax +;; cmovnbeq %rsi, %r10, %r10 +;; movl 0(%r10), %eax ;; jmp label1 ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i32_index_0_guard_yes_spectre_i8_access_0x1000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i32_index_0_guard_yes_spectre_i8_access_0x1000_offset.wat index 7b8d88b480be..70caad3ef762 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i32_index_0_guard_yes_spectre_i8_access_0x1000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i32_index_0_guard_yes_spectre_i8_access_0x1000_offset.wat @@ -44,13 +44,12 @@ ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: ;; movl %edi, %r11d -;; movq %r11, %r9 -;; addq %r9, 0(%rdx), %r9 -;; addq %r9, $4096, %r9 -;; xorq %r10, %r10, %r10 +;; movq 0(%rdx), %rdi +;; lea 4096(%rdi,%r11,1), %r10 +;; xorq %rdi, %rdi, %rdi ;; cmpq $268431359, %r11 -;; cmovnbeq %r10, %r9, %r9 -;; movb %sil, 0(%r9) +;; cmovnbeq %rdi, %r10, %r10 +;; movb %sil, 0(%r10) ;; jmp label1 ;; block1: ;; movq %rbp, %rsp @@ -64,15 +63,14 @@ ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: ;; movl %edi, %r11d -;; movq %r11, %r9 -;; addq %r9, 0(%rsi), %r9 -;; addq %r9, $4096, %r9 -;; xorq %r10, %r10, %r10 +;; movq 0(%rsi), %rsi +;; lea 4096(%rsi,%r11,1), %r10 +;; xorq %rsi, %rsi, %rsi ;; cmpq $268431359, %r11 -;; cmovnbeq %r10, %r9, %r9 -;; movzbq 0(%r9), %rax +;; cmovnbeq %rsi, %r10, %r10 +;; movzbq 0(%r10), %rax ;; jmp label1 ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i32_index_0xffffffff_guard_no_spectre_i32_access_0x1000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i32_index_0xffffffff_guard_no_spectre_i32_access_0x1000_offset.wat index 4bf94389cc26..bdab2846dba8 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i32_index_0xffffffff_guard_no_spectre_i32_access_0x1000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i32_index_0xffffffff_guard_no_spectre_i32_access_0x1000_offset.wat @@ -65,4 +65,4 @@ ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i32_index_0xffffffff_guard_no_spectre_i8_access_0x1000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i32_index_0xffffffff_guard_no_spectre_i8_access_0x1000_offset.wat index 3abcc96daa34..8f925ce9c0d8 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i32_index_0xffffffff_guard_no_spectre_i8_access_0x1000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i32_index_0xffffffff_guard_no_spectre_i8_access_0x1000_offset.wat @@ -65,4 +65,4 @@ ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i32_index_0xffffffff_guard_yes_spectre_i32_access_0x1000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i32_index_0xffffffff_guard_yes_spectre_i32_access_0x1000_offset.wat index 22ca45d7360a..068f359bc566 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i32_index_0xffffffff_guard_yes_spectre_i32_access_0x1000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i32_index_0xffffffff_guard_yes_spectre_i32_access_0x1000_offset.wat @@ -65,4 +65,4 @@ ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i32_index_0xffffffff_guard_yes_spectre_i8_access_0x1000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i32_index_0xffffffff_guard_yes_spectre_i8_access_0x1000_offset.wat index 9174a582cd96..818bd94e0e9a 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i32_index_0xffffffff_guard_yes_spectre_i8_access_0x1000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i32_index_0xffffffff_guard_yes_spectre_i8_access_0x1000_offset.wat @@ -65,4 +65,4 @@ ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i64_index_0_guard_yes_spectre_i32_access_0x1000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i64_index_0_guard_yes_spectre_i32_access_0x1000_offset.wat index cb65508f605c..cc461ab9fddb 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i64_index_0_guard_yes_spectre_i32_access_0x1000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i64_index_0_guard_yes_spectre_i32_access_0x1000_offset.wat @@ -43,13 +43,12 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movq %rdi, %r8 -;; addq %r8, 0(%rdx), %r8 -;; addq %r8, $4096, %r8 -;; xorq %r9, %r9, %r9 +;; movq 0(%rdx), %r10 +;; lea 4096(%r10,%rdi,1), %r9 +;; xorq %r10, %r10, %r10 ;; cmpq $268431356, %rdi -;; cmovnbeq %r9, %r8, %r8 -;; movl %esi, 0(%r8) +;; cmovnbeq %r10, %r9, %r9 +;; movl %esi, 0(%r9) ;; jmp label1 ;; block1: ;; movq %rbp, %rsp @@ -62,15 +61,14 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movq %rdi, %r8 -;; addq %r8, 0(%rsi), %r8 -;; addq %r8, $4096, %r8 -;; xorq %r9, %r9, %r9 +;; movq 0(%rsi), %r10 +;; lea 4096(%r10,%rdi,1), %r9 +;; xorq %r10, %r10, %r10 ;; cmpq $268431356, %rdi -;; cmovnbeq %r9, %r8, %r8 -;; movl 0(%r8), %eax +;; cmovnbeq %r10, %r9, %r9 +;; movl 0(%r9), %eax ;; jmp label1 ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i64_index_0_guard_yes_spectre_i8_access_0x1000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i64_index_0_guard_yes_spectre_i8_access_0x1000_offset.wat index d706afe92e85..ed9ca6fad8f9 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i64_index_0_guard_yes_spectre_i8_access_0x1000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i64_index_0_guard_yes_spectre_i8_access_0x1000_offset.wat @@ -43,13 +43,12 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movq %rdi, %r8 -;; addq %r8, 0(%rdx), %r8 -;; addq %r8, $4096, %r8 -;; xorq %r9, %r9, %r9 +;; movq 0(%rdx), %r10 +;; lea 4096(%r10,%rdi,1), %r9 +;; xorq %r10, %r10, %r10 ;; cmpq $268431359, %rdi -;; cmovnbeq %r9, %r8, %r8 -;; movb %sil, 0(%r8) +;; cmovnbeq %r10, %r9, %r9 +;; movb %sil, 0(%r9) ;; jmp label1 ;; block1: ;; movq %rbp, %rsp @@ -62,15 +61,14 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movq %rdi, %r8 -;; addq %r8, 0(%rsi), %r8 -;; addq %r8, $4096, %r8 -;; xorq %r9, %r9, %r9 +;; movq 0(%rsi), %r10 +;; lea 4096(%r10,%rdi,1), %r9 +;; xorq %r10, %r10, %r10 ;; cmpq $268431359, %rdi -;; cmovnbeq %r9, %r8, %r8 -;; movzbq 0(%r8), %rax +;; cmovnbeq %r10, %r9, %r9 +;; movzbq 0(%r9), %rax ;; jmp label1 ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i64_index_0xffffffff_guard_yes_spectre_i32_access_0x1000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i64_index_0xffffffff_guard_yes_spectre_i32_access_0x1000_offset.wat index 9758a1e0c8ae..5917c8e75587 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i64_index_0xffffffff_guard_yes_spectre_i32_access_0x1000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i64_index_0xffffffff_guard_yes_spectre_i32_access_0x1000_offset.wat @@ -43,13 +43,12 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movq %rdi, %r8 -;; addq %r8, 0(%rdx), %r8 -;; addq %r8, $4096, %r8 -;; xorq %r9, %r9, %r9 +;; movq 0(%rdx), %r10 +;; lea 4096(%r10,%rdi,1), %r9 +;; xorq %r10, %r10, %r10 ;; cmpq $268431356, %rdi -;; cmovnbeq %r9, %r8, %r8 -;; movl %esi, 0(%r8) +;; cmovnbeq %r10, %r9, %r9 +;; movl %esi, 0(%r9) ;; jmp label1 ;; block1: ;; movq %rbp, %rsp @@ -62,15 +61,14 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movq %rdi, %r8 -;; addq %r8, 0(%rsi), %r8 -;; addq %r8, $4096, %r8 -;; xorq %r9, %r9, %r9 +;; movq 0(%rsi), %r10 +;; lea 4096(%r10,%rdi,1), %r9 +;; xorq %r10, %r10, %r10 ;; cmpq $268431356, %rdi -;; cmovnbeq %r9, %r8, %r8 -;; movl 0(%r8), %eax +;; cmovnbeq %r10, %r9, %r9 +;; movl 0(%r9), %eax ;; jmp label1 ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret diff --git a/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i64_index_0xffffffff_guard_yes_spectre_i8_access_0x1000_offset.wat b/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i64_index_0xffffffff_guard_yes_spectre_i8_access_0x1000_offset.wat index 29284edfc23b..16dd75e526cf 100644 --- a/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i64_index_0xffffffff_guard_yes_spectre_i8_access_0x1000_offset.wat +++ b/cranelift/filetests/filetests/isa/x64/wasm/load_store_static_kind_i64_index_0xffffffff_guard_yes_spectre_i8_access_0x1000_offset.wat @@ -43,13 +43,12 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movq %rdi, %r8 -;; addq %r8, 0(%rdx), %r8 -;; addq %r8, $4096, %r8 -;; xorq %r9, %r9, %r9 +;; movq 0(%rdx), %r10 +;; lea 4096(%r10,%rdi,1), %r9 +;; xorq %r10, %r10, %r10 ;; cmpq $268431359, %rdi -;; cmovnbeq %r9, %r8, %r8 -;; movb %sil, 0(%r8) +;; cmovnbeq %r10, %r9, %r9 +;; movb %sil, 0(%r9) ;; jmp label1 ;; block1: ;; movq %rbp, %rsp @@ -62,15 +61,14 @@ ;; movq %rsp, %rbp ;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 } ;; block0: -;; movq %rdi, %r8 -;; addq %r8, 0(%rsi), %r8 -;; addq %r8, $4096, %r8 -;; xorq %r9, %r9, %r9 +;; movq 0(%rsi), %r10 +;; lea 4096(%r10,%rdi,1), %r9 +;; xorq %r10, %r10, %r10 ;; cmpq $268431359, %rdi -;; cmovnbeq %r9, %r8, %r8 -;; movzbq 0(%r8), %rax +;; cmovnbeq %r10, %r9, %r9 +;; movzbq 0(%r9), %rax ;; jmp label1 ;; block1: ;; movq %rbp, %rsp ;; popq %rbp -;; ret \ No newline at end of file +;; ret