IBM Research finds harness changes CUGA task rates 30–81%

IBM Research evaluated three agent harnesses—CUGA, Hermes, and Deep Agents—using the same open-weight model on 90 AppWorld tasks to determine how much the harness impacts task completion. The findings show that task completion rates varied from 30% to 81% based solely on the harness, with CUGA demonstrating stronger consistency for enterprise workflows by enforcing business rules at the runtime level rather than relying on system prompts. Open article →