Skip to content

Latest commit

 

History

History
26 lines (13 loc) · 938 Bytes

File metadata and controls

26 lines (13 loc) · 938 Bytes

More Efficient Multi-Head Attention Implementations

Summary

The figures below summarize the performance benchmarks (lower is better).

 

Forward pass only

 

Forward and backward pass

 

Forward and backward pass after compilation