Anyway to quantize this further down to 4090 level (24Gb VRAM), at Q2_K.gguf level already not sure if it is possible
#1
by askyforever - opened
Thanks!
askyforever changed discussion status to closed
It's possible but you're not going to have a good time