Author finds Nemotron 3.5 Super VL benchmark skewed by SWE-bench contamination

An author testing the NVIDIA Nemotron 3.5 Super VL model against Qwen3.8-Flash-Next on SWE-bench Verified tasks found that benchmark figures can be distorted by data contamination where models have been trained on public datasets. Despite performing poorly on complex coding tasks due to this and other limitations, the Nemotron model proved to be extremely fast and effective for general office research and handling simulated coding agent workloads. Open article →