Frontier quality at 1% of the cost: a production benchmark of LLM classifiers

Airbnb engineers benchmarked a fine-tuned 2-billion-parameter open-weight model against frontier APIs for a production policy classification task. They found that fine-tuning matched frontier-level classification quality while achieving sub-100ms latency and running at roughly 1% of the cost at high scale. The case study demonstrates that fine-tuning domain-specific data can outperform prompt-tuned proprietary models on latency-tight, high-volume workloads. Open article →