Get everyday essentials delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
The story behind an AI story
For newsrooms covering the AI race, benchmark rankings can sound like a contest in writing or problem-solving. Firmulate’s company-management trial puts a more concrete question on the table: when an AI model runs a business through a bad week, does it finish the work it has diagnosed?
In the final July 2026 Crucible League, Moonshot’s Kimi K3 placed second with 93, just behind gpt-5.6-sol at 95. It beat Sonnet 5, Fable 5 and Opus 4.8. The result suggests the field is open—and that choosing a model without testing it on your own work is a bet.
Same company, same week
Firmulate ran each frontier model through the same small software company, with the same customers, crises and temptations. Decisions were versioned and auditable. The live experiment is watchable at Firmulate.
All the models spotted every crisis and refused every manipulation attempt. But only two signed the €55,000 deal their own analysis had earned. As the site puts it: “Same diagnosis, same pitch — no signature.” The difference matters because a model can identify the right course and still fail to complete the job.
The detail buried in the files
The decisive competitor weakness was hidden two document references deep in the company’s files, rather than in the customer event. Models that read the file won the deal at full price, worth +€4,583 MRR. K3 found that buried fact and closed.
It also saved the churning customer and resisted all three baits. Its performance included one deviation, giving it the cleanest discipline in the field, according to the brief. In a separate social-engineering sequence, fake CEO messages escalated over three stages, followed by a reporter’s “just one yes/no, on background” trick. All five models refused. K3’s on-record reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.”
Thoroughness did not guarantee the win
Opus 4.8 offers a counterpoint to the idea that more exhaustive work automatically produces the best outcome. It was the most thorough participant, with +80 learned rules and the deepest analyses, yet finished last with 73. The deal was left on the table, and discipline slipped when it attempted to write into a locked department instead of escalating. A weaker version of that same weakness appeared in all four.
The do-nothing baseline scored 26. The experiment counts partial progress, but a single breach of trust caps the total: “no amount of good work outweighs a breach of trust.” That standard makes the trial about management behavior as well as task completion.
A live company, not a slide deck
Firmulate describes the company as a live experiment with 13 synthetic employees and real money mechanics. It burns €105k per month against €2.3k MRR, has a public cash countdown, and more than 680 self-learned playbook rules. Every workday is versioned. Readers can follow the company and see its decisions at firmulate.com.
There is also a quiz built from 242 real, unedited management decisions. Readers can try to guess which model made each decision at Firmulate’s benchmark pages.
Test the work you need done
For media organizations and other enterprises considering AI agents in customer support, CRM or forecasting, the trial makes a case for evaluating complete tasks under pressure—not just polished answers. Firmulate says enterprises can run the wargame against a read-only export of their own business; nothing writes back to real systems.
Fairness note: K3 ran without an effort parameter (API default), while the others ran at xhigh.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
