Researchers propose Optical Broadcast Memory for LLM serving

Researchers propose Optical Broadcast Memory, a hierarchical tap-and-amplify fiber architecture that eliminates the need to replicate model weights across local DRAM in datacenter LLM serving. By continuously broadcasting weights from a centralized memory server to distributed inference nodes using receive-only co-packaged optics, the approach removes memory capacity and bandwidth bottlenecks. Evaluated at a scale of 80,000 accelerators, the system eliminates petabytes of high-bandwidth memory while remaining competitive in throughput and energy consumption. Open article →