Pluralis: Decentralized AI Model Training and Research Updates
Researchers have adapted open-weight models across decentralized GPUs without architectural changes, using masked activations and compressed synchronization with occasional unmasked passes to correct gradients. The training method matches uncompressed training at over 40 times the throughput on standard internet links and introduces several improvements for billion-parameter models. Open article →