Recommended Samplers/Other Settings?
I'm dipping my toes into locally-hosted AI for roleplaying, using Koboldcpp as my interface of choice. This model is one of the ones that caught my attention, but there doesn't seem to be any recommended sampler values or other settings documented on the main description. Is that something you would be willing to add to the model card? Or are the optimal settings for this model meant to be directly comparable to another model with that info already outlined?
If it helps, I've downloaded the Q6_K version.
My current settings I recommend for 7B/8B/12B/24B models are here
https://huggingface.co/Naphula/Goetia-24B-v1.2
In particular temp 1 and topnsigma 1.25 seems to help a lot with improving quality but sometimes you need to reswipe.
Some models are more sensitive to Rep Pen settings, 1.12 is another, you may need to adjust higher for smaller models (like 1.4 if overcooked) or lower for smarter ones.
Not sure about adaptive_p that is a newer setting added to kobold
(bolded kobold non-defaults)
- Temp 1.0
- TopNSigma 1.25
- Min-P 0.1
- Repetition Penalty 1.08
- Top-P 1.0
- Top-K 100
- Top-A 0
- Typical Sampling 1
- Tail-Free Sampling 1
- Presence Penalty 0
- Sampler Seed -1
- Rp.Range 360
- Rp.Slope 0.7
- Smoothing Factor 0
- Smoothing Curve 1
- DynaTemp 0
- Mirostat Mode OFF ("2" enhances creativity but also errors)
- Mirostat Tau 5
- Mirostat Eta 0.1
- DRY Multiplier 0.8
- DRY Base 1.75
- DRY A.Len 2
- DRY L.Len 320
- XTC Threshold 0.1
- XTC Probability 0.08 (The "Anti-Cliche" Shield)
- DynaTemp ON (The "Poor Man's Fading Mirostat")
- Minimum Temperature 0.65
- Maximum Temperature 1.35
- Temperature 1.0
- DynaTemp-Range 0.35
- DynaTemp-Exponent 1
Much appreciated, thank you!