Salesforce AI Research releases Interactive Visual Gym framework

Salesforce AI Research developed Interactive Visual Gym (IVG), a simulation framework designed to evaluate and train visual AI assistants in dynamic, closed-loop environments. The framework recreates real-world workflows with autonomous simulated users that react in real time to an assistant's guidance, addressing the blind spots of static video benchmarks. Testing 14 leading visual AI assistants through IVG revealed significant limitations in task success and guidance consistency, though the simulator was also shown to successfully improve an open-source model's performance. Open article →