amd/gpt-oss-20b-MoE-Quant-W-MXFP4-A-FP8-KV-FP8

• Introduction

  This model was quantized from openai/gpt-oss-20b using AMD-Quark with calibration samples from the Pile dataset.

• Quantization schemes

  • Quantized Layers: All linear layers excluding *self_attn*, *router*, lm_head, i.e., only MoE MLP linear layers are quantized.
  • Weight: quantized using OCP Microscaling (MX) FP4 scheme
  • Activation: quantized using FP8 symmetric per-tensor scheme
  • KV Cache: FP8 symmetric per-tensor

• Quantization script

python examples/torch/language_modeling/llm_ptq/quantize_quark.py \
    --multi_gpu \
    --model_dir openai/gpt-oss-20b \
    --quant_scheme w_mxfp4_a_fp8 \
    --kv_cache_dtype fp8 \
    --exclude_layers "*self_attn*" "*router*" "lm_head" \
    --dataset pileval \
    --num_calib_data 128 \
    --output_dir amd/gpt-oss-20b-MoE-Quant-W-MXFP4-A-FP8-KV-FP8 \
    --model_export hf_format \
    --skip_evaluation

Deployment

This model supports deployment through the vLLM backend. Please ensure PR#29008 has been correctly applied.

Evaluation

This model is evaluated on gpqa_diamond_generative_n_shot and gsm8k_platinum tasks using the lm-evaluation-harness framework with vllm backend.

Model name	Weight	Activation	KV cache	Exclude	gpqa_diamond_generative_n_shot (5)				gsm8k_platinum
					TP1	TP2	TP4	TP8	TP1	TP2	TP4	TP8
openai/gpt-oss-20b	MXFP4	BF16	BF16	-	0.5606	0.5303	0.5657	0.5606	0.9016	0.9024	0.9032	0.8966
amd/gpt-oss-20b-WMXFP4-AFP8-KVFP8 (this one)	MXFP4	FP8	FP8	*self_attn* *router* lm_head	0.5303	0.5556	0.5152	0.5404	0.8999	0.8900	0.8958	0.9098

Disclaimer

This model is intentionally quantized for the vLLM's CI test usage (tests/models/quantization/test_gpt_oss.py). Model performances are not guaranteed to be optimal.

License

Modifications copyright(c) 2026 Advanced Micro Devices,Inc. All rights reserved.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.