K2 Team releases K2-Horizon open language models
The K2 Team introduced K2-Horizon, a fleet of six open-source language models ranging from 0.9B to 375B parameters designed for reasoning and agentic tasks. To enhance performance, the models incorporate Mixture-of-Value Attention, softplus attention gating, and Uno lossless diffusion adapters. The project follows the LLM360 open-source framework, releasing complete development histories, intermediate checkpoints, training code, and synthetic pre-training data pipelines. Open article →