cyankiwi

/

Qwen3.5-27B-AWQ-BF16-INT4

like 49

Follow

cyankiwi 499

I am happy that you found the model to be of good quality. The recent update is due to improvements in my quantization tool and to using a more well-crafted calibration dataset, i.e., cyankiwi/calibration. In short, the updated model should be better than the previous one :)

I am happy that you found the model to be of good quality. The recent update is due to improvements in my quantization tool and to using a more well-crafted calibration dataset, i.e., cyankiwi/calibration. In short, the updated model should be better than the previous one :)

I'm not speaking for the 27B INT4, but the INT8 was perfect as it was. The answers (coding wise) were indistinguishable from the 27B raw weights. And I ran both :)

Hopefully the updated model works just as well.

I am happy that you found the model to be of good quality. The recent update is due to improvements in my quantization tool and to using a more well-crafted calibration dataset, i.e., cyankiwi/calibration. In short, the updated model should be better than the previous one :)

Thanks for quick response!

After running 1 coding tasks, it seems very promising. Needs more tests to say anything serious.

Have you seen this Reddit?:
https://www.reddit.com/r/LocalLLaMA/comments/1rianwb/running_qwen35_27b_dense_with_170k_context_at/
This article took my attention to your excellent work.
I also have 2 RTX3090 and never had so high quality and fast coding AI like this. (I used to run GGUF Q8 but it is far lower quality)

So final words: many thanks :)

Hi! Your Qwen3.5-27B INT4 quant is seriously impressive — I've been running it for a while and the output quality is remarkably close to the full-precision version. I saw on Reddit people raving about it too, and I completely agree. The coding performance especially is outstanding.

I noticed you recently updated the model and mentioned improvements to your quantization tool and the calibration dataset (cyankiwi/calibration). I'd love to learn more about your approach if you're willing to share:

What makes your calibration dataset special? Any specific domains or criteria for data selection?
Is your quantization tool open-source, or is it a private pipeline?
Did you apply any post-processing or fine-tuning steps after the AWQ process?
Thanks again for the incredible work — it's rare to see quantization this well-executed!

Best regards