relaxed fused multiply-add and fused multiply-subtract

[ngzhian]

Note: this instruction proposal is migrated from WebAssembly/simd#79

1. What are the instructions being proposed?

• relaxed f32x4.fma
• relaxed f32x4.fms
• relaxed f64x2.fma
• relaxed f64x2.fms

2. What are the semantics of these instructions?

All the instructions take 3 operands, a, b, c, perform (a * b) + c or -(a * b) + c:

• relaxed f32x4.fma(a, b, c) = (a * b) + c
• relaxed f32x4.fms(a, b, c) = (a * b) + c
• relaxed f64x2.fma(a, b, c) = -(a * b) + c
• relaxed f64x2.fms(a, b, c) = -(a * b) + c

where:

• the intermediate a * b is be rounded first, and the final result rounded again (for a total of 2 roundings), or
• the the entire expression evaluated with higher precision and then only rounded once.

3. How will these instructions be implemented? Give examples for at least
   x86-64 and ARM64. Also provide reference implementation in terms of 128-bit
   Wasm SIMD.

Detailed implementation guidance available at WebAssembly/simd#79, below is an overview

x86/x86-64 with FMA3

• relaxed f32x4.fma = VFMADD213PS
• relaxed f32x4.fms = VFNMADD213PS
• relaxed f64x2.fma = VFMADD213PS
• relaxed f64x2.fms = VFNMADD213PS

ARM64

• relaxed f32x4.fma = FMLA
• relaxed f32x4.fms = FMLS
• relaxed f64x2.fma = FMLA
• relaxed f64x2.fms = FMLS

ARMv7 with FMA (Neon v2)

• relaxed f32x4.fma = VFMA
• relaxed f32x4.fms = VFMS
• relaxed f64x2.fma = VFMA
• relaxed f64x2.fms = VFMS

ARMv7 without FMA (2 rounding)

• relaxed f32x4.fma = VMLA
• relaxed f32x4.fms = VMLS
• relaxed f64x2.fma = VMLA
• relaxed f64x2.fms = VMLS

Note: Armv8-M will require MVE-F (floating point extension)

RISC-V V

• relaxed f32x4.fma = vfmacc.vv
• relaxed f32x4.fms = vfnmsac.vv
• relaxed f64x2.fma = vfmadd.vv
• relaxed f64x2.fms = vfnmsac.vv

simd128

• relaxed f32x4.fma = f32x4.add(f32x4.mul)
• relaxed f32x4.fms = f32x4.sub(f32x4.mul)
• relaxed f64x2.fma = f64x2.add(f64x2.mul)
• relaxed f64x2.fms = f64x2.sub(f64x2.mul)

4. How does behavior differ across processors? What new fingerprinting surfaces will be exposed?

The difference depends on whether hardware supports FMA or not. The dividing line is between newer and older hardware. Newer (Intel Haswell from 2013 onwards, AMD ZEN from 2017, Cortex-A5 since 2011) hardware tends to come with hardware FMA support so we will probably see less and less hardware without FMA

5. What use cases are there?

Many, especially machine learning (neural nets). Fused multiply-add improves accuracy in numerical algorithms, improves floating-point throughput, and reduces register pressures in some cases. An early prototype and evaluation also showed significant speedup on multiple neural-network models.

[ngzhian]

Power ISA has vmaddfp and vnmsubfp.

[ngzhian]

Example of algorithms requiring fused multiply add https://hal.inria.fr/inria-00000895/document (look for Fast2Mult)

[ngzhian]

@yurydelendik brought up a good point at #77 (comment)
fma(x,y,z) should be (x*y)+z, this follows https://en.cppreference.com/w/cpp/numeric/math/fma

This ordering does change the implementation, e.g. on AArch64, f32.qfma(a, b, c) will be FMLA c, a, b.

[Maratyszcza]

I agree with this point, however we'd need to rename fms into Fused Negative-Multiply-Add (fnma), as Fused Multiply-Subtract is supposed to do fms(x, y, z) = (x*y) - z in this notation.

[yurydelendik]

I was under impression that we have fms only to have a negative product added to the total sum -x*y, e.g. during determinant calculations. Is it okay to still have fms(x, y, z) = -(x*y) + z ?

[Maratyszcza]

I'm not suggesting to remove the quasi-fused -(x*y) + z operation, I suggest that it should be called fnma (Fused Negative Multiply-Add) to better reflect the actual operations involved. In my experience, this instruction is most useful in Newton-Raphson iterations.

[ngzhian]

Adding @dtig fyi, for comments.

[dtig]

Keeping qfma(x,y,z) consistent with cpp intrinsics, and the name edits for fnma sgtm. I can make the necessary changes in V8 when merged into the overview doc.

[ngzhian]

fnma seems to be different on different software:

-a * b + c

• fma operation? JuliaLang/julia#6330 (comment)
• https://manual.gromacs.org/current/doxygen/html-full/group__module__simd.xhtml#gae9ec59669729860c9efd1b9c70f3911b too

-(a * b) + c NOTE: this is the one that we are using in our spec

• https://xsimd.readthedocs.io/en/latest/api/basic_functions.html#_CPPv4I00EN5xsimd4fnmaE5batchI1T1AERK5batchI1T1AERK5batchI1T1AERK5batchI1T1AE
• https://agenium-scale.github.io/nsimd/api_fnma.html
• VFNMADD132SD

- (a * b + c)

• https://www.ibm.com/docs/en/aix/7.1?topic=set-fnmadd-fnma-floating-negative-multiply-add-instruction