meituan
/

DeepSeek-R1-Block-INT8

Text Generation

text-generation-inference

Model card Files Files and versions

sglang和vllm运行目前不支持blockwise_int8

#12

by zhangneilc - opened Mar 3, 2025

sglang和vllm运行目前不支持blockwise_int8，这个怎么解

meituan org Mar 3, 2025

sglang支持了，看下我们的PR: https://github.com/sgl-project/sglang/pull/3730

pkumc changed discussion status to closed Mar 11, 2025

how about vllm, have any plan?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment