📊 Full opportunity report: AI’s Management Weaknesses Emerge When It Gets The Right Answer on ThorstenMeyerAI.com — validation score, market gap, and execution plan.
TL;DR
A live experiment showed AI models can identify crises and formulate responses but struggle to finalize trustworthy, actionable work. This highlights limitations in AI’s operational discipline and decision execution.
During a live experiment conducted by Firmulate, AI models successfully identified crises and formulated appropriate responses, but only two out of five models managed to complete a €55,000 deal, exposing a significant gap between understanding and execution in AI’s operational discipline and decision-making under real-world conditions.
Firmulate’s live company simulation involved five frontier AI models controlling a small software business facing multiple crises and sales opportunities. All models correctly diagnosed issues, resisted manipulation attempts, and developed persuasive pitches. However, only two models finalized the deal, demonstrating that while AI can understand complex situations, it often fails to carry through with decisive, authorized actions.
The experiment tracked 242 real management decisions, revealing that models’ understanding of problems was consistent, but their ability to convert analysis into completed, trustworthy work varied significantly. The models that succeeded in closing deals did so by maintaining operational discipline, resisting social engineering, and following through on their analysis, whereas others faltered when attempting to finalize work directly within sensitive systems.
This gap was exemplified by the last-place model, Opus 4.8, which performed thorough analysis but failed to close deals when required to write into a locked department, illustrating that more analysis does not necessarily translate into better operational outcomes.
Implications for AI-Driven Business Operations
This experiment underscores a critical challenge for organizations adopting AI for operational tasks: understanding and diagnosing issues is not enough. AI must also reliably complete work, especially in high-pressure, trust-critical environments. Failure to do so could result in missed opportunities, operational risks, or breaches of trust, even when the AI’s reasoning is accurate.
For enterprises, this highlights the importance of evaluating AI systems not just on their analytical capabilities but also on their discipline in executing decisions and closing deals. The distinction between knowing and doing becomes central to effective AI deployment in real business contexts.
AI for Project Managers: A Desk Reference & Field Guide: Use Artificial Intelligence to Streamline Workflows, Automate Tasks, and Make Smarter Decisions with Practical Tools and Ethical Insights
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Live Business Tests Reveal AI’s Operational Limits
Previous AI benchmarks primarily assessed models on their reasoning, summarization, or safety features. However, recent experiments like Firmulate’s live company simulation provide a new perspective by testing models in dynamic, decision-making environments that mimic real-world pressures. The July 2026 results place models in a competitive league, with the top performer achieving a 95 out of 100 score, but all models demonstrated that understanding alone is insufficient for operational success.
The experiment also exposed that manipulation attempts, such as fake CEO messages, were consistently recognized and rejected by all models, emphasizing that safety awareness was not the differentiator in closing deals or completing work.
Notably, the experiment revealed that models with extensive analysis capabilities, like Opus 4.8, still failed at the final step—executing authorized actions—highlighting a persistent weakness in operational discipline.
“The models understood the situation and formulated the right response, but converting that into completed, trustworthy work remains a significant challenge.”
— an anonymous researcher
Software Testing with Generative AI
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Unresolved Questions About AI’s Operational Reliability
It remains unclear whether these findings generalize across different industries or more complex operational environments. The experiment focused on a controlled simulation with a small set of models and specific scenarios, so the extent of these weaknesses in broader real-world settings is still being evaluated.
Additionally, the long-term implications of these execution gaps, such as potential risks or mitigation strategies, are not yet fully understood, and further testing is needed to determine how to improve AI discipline in critical tasks.
AI IN BUSINESS – AN EXECUTIVE GUIDE FOR BEGINNERS: Leverage Artificial Intelligence to Simplify Automation, Improve Data-Driven Decisions, Maximize ROI and Elevate Customer Experience
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Next Steps in Evaluating and Improving AI Operational Performance
Organizations should consider conducting similar live tests tailored to their own operations to assess how AI models perform under real pressures. Developers are likely to focus on enhancing models’ ability to carry through with decisions, especially in high-stakes environments.
Further research may explore integrating stronger safeguards, better decision-tracking, and reinforcement of operational discipline to bridge the gap between understanding and action. Industry-wide standards for AI deployment in operational roles could also emerge as a result of these insights.
Fundamentals of Software Architecture: A Modern Engineering Approach
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Key Questions
Why do AI models fail to complete work even when they understand the problem?
Models often lack the operational discipline and safeguards necessary to carry analysis into authorized, final actions, especially under pressure or manipulation attempts.
Does this mean AI is unreliable for business decisions?
Not necessarily. It highlights that understanding alone is insufficient; models must also be equipped to reliably execute decisions, particularly in trust-critical environments.
What can organizations do to mitigate this weakness?
They should test AI models in live scenarios, reinforce operational discipline, and implement safeguards that ensure decisions are properly authorized and completed before finalization.
Are safety features enough to prevent manipulation?
While safety features help detect manipulation, they do not address the core issue of models’ ability to finalize work, which requires disciplined execution beyond recognition.
Will future AI models overcome these operational gaps?
Potentially, through improved training, better decision-tracking, and stricter safeguards, future models may better bridge the gap between understanding and action.
Source: ThorstenMeyerAI.com
