Hugging Face's logo
sn2234's picture
9 25

sn2234

sn2234
Ā·

AI & ML interests

None yet

Recent Activity

liked a model about 21 hours ago
zerofata/Q3.5-BlueStar-v2-27B
reacted to SeaWolf-AI's post with šŸ”„ 10 days ago
šŸŸļø Smol AI WorldCup: A 4B Model Just Beat 8B ā€” Here's the Data We evaluated 18 small language models from 12 makers on 125 questions across 7 languages. The results challenge the assumption that bigger is always better. Community Article: https://huggingface.co/blog/FINAL-Bench/smol-worldcup Live Leaderboard: https://huggingface.co/spaces/ginigen-ai/smol-worldcup Dataset: https://huggingface.co/datasets/ginigen-ai/smol-worldcup What we found: ā†’ Gemma-3n-E4B (4B, 2GB RAM) outscores Qwen3-8B (8B, 5.5GB). Doubling parameters gained only 0.4 points. RAM cost: 2.75x more. ā†’ GPT-OSS-20B fits in 1.5GB yet matches Champions-league dense models requiring 8.5GB. MoE architecture is the edge AI game-changer. ā†’ Thinking models hurt structured output. DeepSeek-R1-7B scores 8.7 points below same-size Qwen3-8B and runs 2.7x slower. ā†’ A 1.3B model fabricates confident fake content 80% of the time when prompted with nonexistent entities. Qwen3 family hits 100% trap detection across all sizes. ā†’ Qwen3-1.7B (1.2GB) outscores Mistral-7B, Llama-3.1-8B, and DeepSeek-R1-14B. Latest architecture at 1.7B beats older architecture at 14B. What makes this benchmark different? Most benchmarks ask "how smart?" ā€” we measure five axes simultaneously: Size, Honesty, Intelligence, Fast, Thrift (SHIFT). Our ranking metric WCS = sqrt(SHIFT x PIR_norm) rewards models that are both high-quality AND efficient. Smart but massive? Low rank. Tiny but poor? Also low. Top 5 by WCS: 1. GPT-OSS-20B ā€” WCS 82.6 ā€” 1.5GB ā€” Raspberry Pi tier 2. Gemma-3n-E4B ā€” WCS 81.8 ā€” 2.0GB ā€” Smartphone tier 3. Llama-4-Scout ā€” WCS 79.3 ā€” 240 tok/s ā€” Fastest model 4. Qwen3-4B ā€” WCS 76.6 ā€” 2.8GB ā€” Smartphone tier 5. Qwen3-1.7B ā€” WCS 76.1 ā€” 1.2GB ā€” IoT tier Built in collaboration with the FINAL Bench research team. Interoperable with ALL Bench Leaderboard for full small-to-large model comparison. Dataset is open under Apache 2.0 (125 questions, 7 languages). We welcome new model submissions.
updated a collection 27 days ago
Datasets
View all activity

Organizations

Hugging Face Discord Community's profile picture