Q8 version please
#2
by acediac - opened
Any chance someone could upload an 8bit version? I tried using mlx-my-repo but failed with ernie45_moe not supported because the codebase is too old on that tool.
@acediac Here https://huggingface.co/finding1/ERNIE-4.5-300B-A47B-MLX-8.5bpw. It says 8.5 bpw because that's what the console said but it was run with --q-bits 8.
acediac changed discussion status to closed