Recommended Sampling Parameters
Hi OpenAI Team,
Thank you for your awesome contribution to the open source community! I am wondering what is the recommended sampling parameters for gpt-oss (temperature, top_p, etc)? Does not seem to be mentioned in the readme, generation_config.json, or the official blog. Would appreciate any pointers or best practices you could share!
you can also use the default params in the eval: https://github.com/openai/gpt-oss/blob/89fe402d10a59879781a1eb0a64affdf4c278a4d/gpt_oss/evals/__main__.py#L41
The recommended sampling parameters are temp = 1.0 and top_p = 1
What about top_k ? Is there a recommended setting for it?
@VladNC Based on
- the torch reference implementation (1 and 2),
- the guidance in the readme and
- the vlllm example
I'd say guidance is to disable top_k. Unfortunately, top_k defaults to 50 in transformers lib on huggingface if left unspecified, so be wary of this.
@dkundel-openai I would appreciate if you could confirm my interpretation