Poll: Will 2026 be the year of subquadratic attention?

The transformer architecture is cursed by its computational complexity.
It is why you run out of tokens and have to compact. But some would argue that this is a feature not a bug and that this is also why these models are so good. We've been doing a lot of research on trying to make equally good models that are computationally cheaper, But so far, none of the approaches have stood the test of time. Or so it seems.

Please vote, don't be shy. Remember that the Dunning-Kruger effect is very real, so the person who knows less about transformers than you is going to vote. We want everyone's opinion, no matter confidence.

👍 if you think at least one frontier model* will have no O(n^2) attention by the end of 2026
🔥 If you disagree

* Frontier models - models that match / outperform the flagship claude, gemini or chatgpt at the time on multiple popular benchmarks

4 replies

updated a dataset 26 days ago

fungamer2/CustomThinking

Viewer • Updated about 7 hours ago • 4.41k • 156

New activity in fungamer2/Ami 27 days ago

Title

#5 opened 27 days ago by

fungamer2

New activity in fungamer2/autotrain-advanced-2 28 days ago

Your training has failed ❌

#2 opened 28 days ago by

fungamer2

updated a dataset 28 days ago

fungamer2/General-Thinking

Viewer • Updated 26 days ago • 10k • 49

published a dataset 29 days ago

fungamer2/CustomThinking

Viewer • Updated about 7 hours ago • 4.41k • 156

liked a model about 1 month ago

fungamer2/Ami-360M

0.4B • Updated 22 days ago • 22 • 1

updated a Space about 1 month ago

Ami

💬

Chat with an AI assistant

reacted to marksverdhei's post with 🤗 about 1 month ago

Post

2664

Dear Hugging Face team, can we please have a way to archive hf repositories / spaces? I have a bunch of spaces that used to work but don't any more due to the hf space implementations changing and i think it would be good if I could archive those like in GitHub.

React to this post if you want to see this feature! 💡