A researcher benchmarked Claude's Fable 5 and GPT-5.6 Sol on an unpublished NP‑hard optimization problem, with and without the /goal mode.

The KIRO problem asks solvers to connect distribution points and terminals in a fiber‑network design while minimizing total cable length.

The benchmark used three paired runs for each model under a 30‑minute limit and a 1,900‑second agent timeout.

An initial single run for each model provided a baseline.

The main comparison consisted of three runs for Fable 5 and three for GPT-5.6 Sol.

Fable 5's plain‑mode mean score was 1,875 points lower than Sol's mean.

In goal mode, Fable 5's mean was 1,984 points lower than Sol's, but the overall means for both models were higher than in plain mode.

Fable 5's plain‑mode scores varied by 319 points, while Sol's varied by 1,958 points.

Fable 5 achieved its best goal‑mode score of 31,934; Sol's best plain‑mode score was 32,703.

Negative differences were observed in four of six trials, indicating that /goal performed better in those cases.

Analysis showed that /goal works differently in Claude Code, where a session‑scoped evaluator monitors the goal, and in Codex, where the goal is stored as persistent thread state with tool access.

Limitations include the use of a single unpublished task, limited model replication, and possible resource allocation differences due to container CPU allocations.

Reproduction scripts and data are available in the CLIArena repository, allowing others to run the benchmark and review detailed logs.

The results suggest that persistence features may improve individual trial outcomes but can reduce average performance on hard optimization problems.