Architecture Decisions for Marin 535B-A23B | Open Athena

The author details the architecture, reasoning, and pretraining experiments behind Marin 535B-A23B, a large mixture-of-experts model trained on 18 trillion tokens. The post examines design decisions driven by hardware utilization on GB200 systems, stability goals, and risk management through scaling ladders. Open article →