| --- |
| license: apache-2.0 |
| tags: |
| - slidesparse |
| - sparse |
| - quantization |
| - int8 |
| - fp8 |
| - llama |
| - qwen |
| --- |
| |
| # SlideSparse Checkpoints |
|
|
| Pre-converted sparse model checkpoints using the **SlideSparse** technique. |
|
|
| ## Overview |
|
|
| This repository contains model weights converted with various sparsity configurations: |
| - **2:4** - Standard N:M sparsity (50% sparse) |
| - **2:6** - Extended sparsity (67% sparse) |
| - **2:8** - Higher sparsity (75% sparse) |
| - **2:10** - Maximum sparsity (80% sparse) |
|
|
| ## Models Included |
|
|
| | Base Model | Quantization | Sparsity Variants | |
| |------------|--------------|-------------------| |
| | Llama-3.2-1B | INT8, FP8 | 2:4, 2:6, 2:8, 2:10 | |
| | Llama-3.2-3B | INT8, FP8 | 2:4, 2:6, 2:8, 2:10 | |
| | Qwen2.5-7B | INT8, FP8 | 2:4, 2:6, 2:8, 2:10 | |
| | Qwen2.5-14B | INT8, FP8 | 2:4, 2:6, 2:8, 2:10 | |
|
|
| ## Source Models |
|
|
| These checkpoints are derived from: |
| - [RedHatAI/Llama-3.2-1B-Instruct-quantized.w8a8](https://huggingface.co/RedHatAI/Llama-3.2-1B-Instruct-quantized.w8a8) |
| - [RedHatAI/Llama-3.2-3B-Instruct-quantized.w8a8](https://huggingface.co/RedHatAI/Llama-3.2-3B-Instruct-quantized.w8a8) |
| - [RedHatAI/Qwen2.5-7B-Instruct-quantized.w8a8](https://huggingface.co/RedHatAI/Qwen2.5-7B-Instruct-quantized.w8a8) |
| - [RedHatAI/Qwen2.5-14B-Instruct-quantized.w8a8](https://huggingface.co/RedHatAI/Qwen2.5-14B-Instruct-quantized.w8a8) |
|
|
| ## License |
|
|
| - **Qwen models**: Apache 2.0 |
| - **Llama models**: Please refer to [Meta's Llama license](https://llama.meta.com/llama3/license/) |
|
|
| ## Usage |
|
|
| ```bash |
| # Download all checkpoints |
| huggingface-cli download bcacdwk/slidesparse-checkpoints --local-dir ./checkpoints_slidesparse |
| |
| # Download specific model |
| huggingface-cli download bcacdwk/slidesparse-checkpoints Llama3.2-1B-INT8-SlideSparse-2_4 --local-dir ./checkpoints_slidesparse/Llama3.2-1B-INT8-SlideSparse-2_4 |
| ``` |
|
|
| ## Citation |
|
|
| If you use these checkpoints, please cite the SlideSparse paper (coming soon). |
|
|
