LocoOperator-4B-GGUF

This repository contains the official GGUF quantized versions of LocoOperator-4B.

LocoOperator-4B is a 4B-parameter code exploration agent distilled from Qwen3-Coder-Next. It is specifically optimized for local agent loops (like Claude Code style), providing high-speed codebase navigation with 100% JSON tool-calling validity.

🚀 Which file should I choose?

We provide several quantization levels to balance performance and memory usage:

File Name	Size	Recommendation
LocoOperator-4B.Q8_0.gguf	4.28 GB	Best Accuracy. Recommended for local agent loops to ensure perfect JSON output.
LocoOperator-4B.Q6_K.gguf	3.31 GB	Great Balance. Near-lossless logic with a smaller footprint.
LocoOperator-4B.Q4_K_M.gguf	2.50 GB	Standard. Compatible with almost all local LLM runners (LM Studio, Ollama, etc.).
LocoOperator-4B.IQ4_XS.gguf	2.29 GB	Advanced. Uses Importance Quantization for better performance at smaller sizes.

🛠 Usage (llama.cpp)

To run this model using llama-cli or llama-server, we recommend a context size of at least 50K to handle multi-turn codebase exploration:

Simple CLI Chat:

./llama-cli \
    -m LocoOperator-4B.Q8_0.gguf \
    -c 51200 \
    -p "You are a helpful codebase explorer. Use tools to help the user."

Serve as an OpenAI-compatible API:

./llama-server \
    -m LocoOperator-4B.Q8_0.gguf \
    --ctx-size 51200 \
    --port 8080

📋 Model Details

• Base Model: Qwen3-4B-Instruct-2507
• Teacher Model: Qwen3-Coder-Next
• Training Method: Full-parameter SFT (Knowledge Distillation)
• Primary Use Case: Codebase exploration (Read, Grep, Glob, Bash, Task)

🔗 Links

• Main Repository: LocoreMind/LocoOperator-4B
• GitHub: LocoreMind/LocoOperator
• Blog: locoremind.com/blog/loco-operator

🙏 Acknowledgments

Special thanks to mradermacher for the initial quantization work and the llama.cpp community.