The big downside of this approach is that to you have to finetune llm each time you want to add a new tool, and finetuning is complicated. It's much easier to use special languages like sglang or guidance. And if you are restricted to use small llm, you always may finetune it the regular way and use with sglang. Actually, it would be interesting to see which model would win: Toolformer or regular finetuned transformer + sglang
Totally agree! I think it's a good framework for thinking about how an LLM could learn to use tools, but to be practical in reality you need to allow it to pick arbitrary tools from a codebase or toolchain without fine-tuning each time.
The big downside of this approach is that to you have to finetune llm each time you want to add a new tool, and finetuning is complicated. It's much easier to use special languages like sglang or guidance.
And if you are restricted to use small llm, you always may finetune it the regular way and use with sglang. Actually, it would be interesting to see which model would win: Toolformer or regular finetuned transformer + sglang
Totally agree! I think it's a good framework for thinking about how an LLM could learn to use tools, but to be practical in reality you need to allow it to pick arbitrary tools from a codebase or toolchain without fine-tuning each time.