Tool calls destroy good reasoning + how to fix it
#25
by ffewfqefwsefwfe - opened
llama-server UI reference (doesn't have tool calls):
Shows good reasoning.
Open WebUI without tool calls:
When you disable all tools, the reasoning matches what we see in llama-server UI.
Open WebUI with tool calls:
As you can see, the good reasoning is gone as soon as we use tools.
Fix:
Added proxy middleware (python fastapi service) to make tool-calls a 2-step process:
- strip tools from request, analyze query & create a plan
- re-attach tools and feed good reasoning into 2nd LLM call
Result:
As you can see it now uses good reasoning + tool-calling.
PS:
For OpenCode I also had to drop the temperature to 0.5. Much better results.