NVIDIA NeMo Switchyard routing increases token costs

An AI engineer experimenting with NVIDIA's NeMo Switchyard model-routing tool finds that model routing can easily increase token usage and costs rather than saving them. Testing different configurations with SWE-bench coding tasks revealed that LLM classifiers often fail to select cheaper local models, and weaker models failing on hard tasks inflate context sizes for stronger escalation models. Open article →