Hugging Face's logo

Sahabat-AI
/
Llama-Sahabat-AI-v2-70B-IT

Text Generation
Transformers
Safetensors
llama
conversational
text-generation-inference
Model card Files
xet
 Community
3

Quantization

#1
by hadadrjt - opened
Discussion
hadadrjt

Are there any plans for quantization, such as 2-bit and 4-bit with Ollama? This could reduce resource usage.
andreaschandra

+1
andreaschandra

@hadadrjt have you tried to quantize the model by yourself?
hadadrjt

Yeah
andrebagus

@andreaschandra waddap, its me
andreaschandra

@hadadrjt how did u do it?
budikomarudin
•
edited Jun 17, 2025

@hadadrjt i'v push 4-bit quantization version here https://ollama.com/csalab/sahabatai2/tags
andreaschandra

@budikomarudin great! I wonder if there is some technical blog post to repro?
budikomarudin

@andreaschandra I tried converting it to GGUF and quantizing it using Llama.cpp (Brew).

Sign up or log in to comment