NVIDIA releases Reshard collective for RL weight refit

Nvidia developers have introduced the Reshard collective, a new type of communication abstraction designed to accelerate reinforcement learning weight refit by up to 5.7x for trillion-parameter models. By reformulating weight refit as a PyTorch distributed tensor reshard problem, the open-source NCCL M2N Reshard implementation transfers only the theoretical minimum amount of weight data between training and inference GPUs. A reference implementation is available in the NVIDIA NeMo RL framework. Open article →