Skip to content

Issues: tomaarsen/attention_sinks

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

18 Open 12 Closed
18 Open 12 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

TypeError: bad operand type for unary -: 'NoneType'
#47 opened Aug 7, 2024 by wln20
Last generated token getting ignored in streaming.py?
#45 opened Mar 29, 2024 by ritik99
TypeError: 'NoneType' object is not subscriptable
#43 opened Jan 16, 2024 by Kuchiriel
Support newer versions of mistral (e.g. mistralai/Mistral-7B-Instruct-v0.2)?
#41 opened Jan 10, 2024 by spring1915
2
chatglm3 support?
#40 opened Dec 26, 2023 by ScottishFold007
3.3: Learnable Sink Token
#38 opened Dec 20, 2023 by photomz
1
KeyError: 'Cache only has 0 layers, attempted to access layer with index 0'
#37 opened Dec 18, 2023 by pseudotensor
8
Error when using Qwen 7b chat
#36 opened Dec 11, 2023 by Minami-su
1
Error loading Qwen-1_8B
#35 opened Dec 4, 2023 by haiphong93
Generation stop；torch.cuda.OutOfMemoryError: CUDA out of memory.
#34 opened Nov 24, 2023 by Essence9999
GPTQ models support
#31 opened Nov 20, 2023 by synacktraa
5
Flash Attention Support
#30 opened Nov 17, 2023 by Jiayuanhip
1
Avoid overly strict "transformers==4.34.0",
#26 opened Oct 24, 2023 by pseudotensor
2
Error when using Qwen-14B
#24 opened Oct 23, 2023 by sun1092469590
16
Issue with only adding sink tokens in cache
#17 opened Oct 12, 2023 by sam1373
4
Experiments with MPT7b with seqlen > 2048
#14 opened Oct 10, 2023 by vchiley
4
Add benchmarks comparing against Sliding Window Attention
#10 opened Oct 5, 2023 by casper-hansen
1
Add cotributing.md
#9 opened Oct 5, 2023 by rajveer43
ProTip! Adding no:label will show everything without a label.