Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers
Paper
• 2311.10642 • Published
• 25
Danilo Dordevic
danilodjor
AI & ML interests
Deep learning, Vision Transformers, GNNs
Recent Activity
liked
a model 12 days ago
Qwen/Qwen3-VL-2B-Instruct liked
a model 12 days ago
AIDC-AI/Ovis-U1-3B authored
a paper
over 2 years ago
Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as
an Alternative to Attention Layers in Transformers Organizations
None yet
authored a paper over 2 years ago