Evaluation of AI company-brain harnesses on mock Slack tasks

An evaluation of seven AI harnesses tested on office work tasks using a mock Slack integration revealed that pass rates across models were closely grouped. Hermes using Opus topped the chart with a success rate of 78.8 percent at $0.24 per solved task. The evaluation also tracked cost per solved task and identified a common failure mode where harnesses missed corrections posted in different channels. Open article →