g023/qwen3-tiny-v2
Text Generation • 2B • Updated
g023dev
g023
AI & ML interests
ai datasets, ai training,ai software
Recent Activity
updated a model about 4 hours ago
g023/qwen3-tiny-v2 updated a model about 4 hours ago
g023/qwen3-tiny-v1 published a model about 4 hours ago
g023/qwen3-tiny-v2Organizations
None yet
updated 2
models about 4 hours ago
published 2
models about 4 hours ago
repliedto DedeProGames's post 10 days ago
If done in small pieces, and with good memory management, they can manage pretty decently.
reactedto DedeProGames's post with 🤗 10 days ago
Post
3823
Can small models program?
Although even if they are reasoning AIs, small AIs cannot create extensive and high-quality code, at least that's what is commonly thought.
We present OrionLLM/NanoCoder-0.6b, an AI with just 600 million parameters based on qwen3-0.6b and trained with the dataset nvidia/OpenCodeReasoning.
While not good at complex code, we observed a significant improvement in code generation (especially in Python code), demonstrating that, when trained correctly, small AIs can, in fact, program.
Although even if they are reasoning AIs, small AIs cannot create extensive and high-quality code, at least that's what is commonly thought.
We present OrionLLM/NanoCoder-0.6b, an AI with just 600 million parameters based on qwen3-0.6b and trained with the dataset nvidia/OpenCodeReasoning.
While not good at complex code, we observed a significant improvement in code generation (especially in Python code), demonstrating that, when trained correctly, small AIs can, in fact, program.
- 2 replies
updated a
model 11 days ago
g023/Qwen3-1.77B-g023-GGUF
🤗 1
#8645 opened 16 days ago
by g023
published a
model 16 days ago
updated a
model 16 days ago
published a
model 16 days ago
updated a
model 16 days ago
published a
model 16 days ago
reactedto OzTianlu's post with 🤗 about 1 month ago
Post
3462
O(1) inference is the foundational design of Spartacus-1B-Instruct 🛡️ !
NoesisLab/Spartacus-1B-Instruct
We have successfully replaced the KV-cache bottleneck inherent in Softmax Attention with Causal Monoid State Compression. By defining the causal history as a monoid recurrence, , the entire prefix is lossily compressed into a fixed-size state matrix per head.
The technical core of this architecture relies on the associativity of the monoid operator:
Training: parallel prefix scan using Triton-accelerated JIT kernels to compute all prefix states simultaneously.
Inference: True sequential updates. Memory and time complexity per token are decoupled from sequence length.
Explicit Causality: We discard RoPE and attention masks. Causality is a first-class citizen, explicitly modeled through learned, content-dependent decay gates.
Current zero-shot benchmarks demonstrate that Spartacus-1B-Instruct (1.3B) is already outperforming established sub-quadratic models like Mamba-1.4B and RWKV-6-1.6B on ARC-Challenge (0.3063). Recent integration of structured Chain-of-Thought (CoT) data has further pushed reasoning accuracy to 75%.
The "Spartacus" era is about scaling intelligence, not the memory wall ♾️.
NoesisLab/Spartacus-1B-Instruct
We have successfully replaced the KV-cache bottleneck inherent in Softmax Attention with Causal Monoid State Compression. By defining the causal history as a monoid recurrence, , the entire prefix is lossily compressed into a fixed-size state matrix per head.
The technical core of this architecture relies on the associativity of the monoid operator:
Training: parallel prefix scan using Triton-accelerated JIT kernels to compute all prefix states simultaneously.
Inference: True sequential updates. Memory and time complexity per token are decoupled from sequence length.
Explicit Causality: We discard RoPE and attention masks. Causality is a first-class citizen, explicitly modeled through learned, content-dependent decay gates.
Current zero-shot benchmarks demonstrate that Spartacus-1B-Instruct (1.3B) is already outperforming established sub-quadratic models like Mamba-1.4B and RWKV-6-1.6B on ARC-Challenge (0.3063). Recent integration of structured Chain-of-Thought (CoT) data has further pushed reasoning accuracy to 75%.
The "Spartacus" era is about scaling intelligence, not the memory wall ♾️.
updated a
model about 2 months ago
upvoted a paper about 2 months ago
published a
model 2 months ago
updated a
model 2 months ago