0.3.3

What's Changed

• chore: update flash attention config by @eegli in #18

New Contributors

• @eegli made their first contribution in #18

Full Changelog: 0.3.2...0.3.3

0.3.2

Full Changelog: 0.3.1...0.3.2

0.3.1

Full Changelog: 0.3.0...0.3.1

0.3.0

Full Changelog: 0.2.1...0.3.0

0.2.1

make sure it supports greater than 2 hierarchies

0.2.0

move closer to what the paper did, with local and global token embedd…

…ings not shared

0.1.7

switch to rotary embeddings, as they did in the paper

0.1.6

evidence is emerging that decoders generate implicit absolute and rel…

…ative positions without help

0.1.4

address https://github.com/lucidrains/MEGABYTE-pytorch/issues/10

0.1.2

learn on the very first start token