use vllm==0.6.3 load this model，it generate the fowllowing error
#1
by wc-llm - opened
when I use vllm==0.6.3 load this model，it generate the fowllowing error
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/parameter.py", line 133, in load_qkv_weight
assert param_data.shape == loaded_weight.shape
AssertionError
wc-llm changed discussion status to closed