Databricks CTO Matei Zaharia said the company undertook the evaluation because existing benchmark tests such as SWE‑Bench are broken. He noted that cheaper per‑token pricing does not imply cheaper per‑task. For example, Sonnet 5 costs less per token than Opus 4.8 but uses more tokens, resulting in higher cost and lower quality. The benchmark measured task success rates, finding that GLM 5.2 matched Opus 4.8 on quality while costing $1.28 per task versus $1.94 for Opus. Academics observed a similar pattern in March, noting that in roughly one third of comparisons the lower listed price corresponded with higher overall cost. Zaharia also reported that the harness software influences cost and performance. The simple Pi harness achieved the same success rate as vendor harnesses for Opus and GPT 5.5 but at half the cost. Context size differed: Pi used about 236,999 tokens per task, whereas Opus with Claude Code required 742,000 tokens, about three times more. Codex required 1,235,000 tokens per task compared to 665,000 for Pi. Databricks built Omnigent, a wrapper that combines and swaps multiple coding agents, to facilitate harness flexibility.