experimental 4-bit dynamic quant using unsloth, these model will be used for inference and RoPE fine-tuning.
huggingkot
huggingkot
AI & ML interests
None yet
Organizations
None yet
-
huggingkot/Qwen2.5-Coder-14B-Instruct-q4f16_1-MLC
Text Generation • Updated -
huggingkot/Qwen2.5-Coder-7B-Instruct-q4f16_1-MLC
Text Generation • Updated -
huggingkot/Qwen2.5-Coder-3B-Instruct-q4f16_1-MLC
Text Generation • Updated -
huggingkot/Qwen2.5-Coder-1.5B-Instruct-q4f16_1-MLC
Text Generation • Updated
experimental 4-bit dynamic quant using unsloth, these model will be used for inference and RoPE fine-tuning.
for model less than 8b
-
huggingkot/Qwen2.5-Coder-14B-Instruct-q4f16_1-MLC
Text Generation • Updated -
huggingkot/Qwen2.5-Coder-7B-Instruct-q4f16_1-MLC
Text Generation • Updated -
huggingkot/Qwen2.5-Coder-3B-Instruct-q4f16_1-MLC
Text Generation • Updated -
huggingkot/Qwen2.5-Coder-1.5B-Instruct-q4f16_1-MLC
Text Generation • Updated
https://llm.mlc.ai/docs
bookmark