Together Compiler: Continuous Compilation for AI inference

Together AI has introduced the Together Compiler, a continuous compilation system designed to optimize AI inference workloads across diverse hardware and dynamically changing traffic shapes. The company reports that the compiler uses high-throughput persistent agents and a custom cost model to achieve significantly higher token-per-second performance compared to existing baselines while generalizing to domains beyond language models. Open article →