OpenRouter publishes LLM benchmarking study and set

OpenRouter published a tutorial and benchmark study explaining how to build code-checked evaluation sets for large language models, testing tasks across multiple models. The authors found that Sol achieved a 100 percent pass rate on their 60 synthetic tasks while DeepSeek offered the lowest response cost, and noted that agreement with public rankings was weak. Open article →