Skip to content

Latest commit

 

History

History
 
 

02_bonus_efficient-multihead-attention

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 

More Efficient Multi-Head Attention Implementations

Summary

The figures below summarize the performance benchmarks (lower is better).

 

Forward pass only

 

Forward and backward pass

 

Forward and backward pass after compilation