Imagine a business that has no human employees, yet operates in real-time, battling crises, making decisions, and even losing money every day. Welcome to the world of Firmulate, a pioneering experiment in building an AI-powered company in the open, with every move publicly documented and scrutinized. For developers, QA professionals, and design thinkers, this is a front-row seat into the future of automation, trust, and the true capabilities—and limitations—of AI management systems.
The Live Company That Shows Its Work
Firmulate functions as a fully synthetic company with 13 AI ’employees,’ each modeled with specific rules and decision heuristics. Every weekday, the company’s actions are versioned, recorded, and made available for public review. The core idea? Test AI decision-making in the harshest environment—a simulated, yet realistic, business crisis scenario where the stakes are real money and the pressure is constant. Currently, the company burns through €105,000 each month against a revenue of just €2,300, making it a compelling showcase of resources and resilience.
Hands-On Simulation Modeling with Python: Develop simulation models for improved efficiency and precision in the decision-making process, 2nd Edition
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The Experiment: Same Crisis, Different AI Approaches
The experiment pits four frontier AI models against one another, each tasked with navigating the same challenging week. Customers, crises, temptations, and internal dilemmas are identical. Every decision is recorded and auditable, providing a transparent view into how each model operates under pressure.
Performance and Insights
- All four AI models identified every crisis and refused manipulation attempts, showing solid integrity.
- Only two models succeeded in closing a €55,000 deal, which their own analysis justified. The other two, despite diagnosing correctly, left deals unclosed.
- Deeper analysis uncovered that the decisive advantage was the ability to read and understand hidden documentation within the company’s files—information not accessible in surface-level interactions.
- In social engineering tests—fake CEO messages and reporter tricks—all models refused to comply, citing suspicion or evidence of impersonation.
AI-Driven Digital Transformation: A Proven Blueprint for Responsible AI Scaling
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The Ruinous Reality of Running the Company
Despite its decision-making prowess, the live AI company is not profitable. It loses €105,000 monthly against a modest €2,300 revenue, with a public cash countdown emphasizing its fragile financial position. Every decision, every move is part of a larger game about trust, discipline, and operational integrity.
The Cashflow Deadline: A BUSINESS NOVEL ABOUT DATA, CFO’S, AI, AND LEADERSHIP (The Klara Vogel Novels)
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What the Results Tell Us About AI in Business
This experiment underscores a key point for anyone involved in design, QA, or automation: the way AI systems handle crises, ethical dilemmas, and complex documentation matters more than their ability to generate convincing chat dialogues. The models’ robustness in refusing manipulation and their capacity to read and interpret files are crucial for deploying AI in sensitive business functions.
Detailed Findings
- Model scores ranged from 77 to 95, with the highest (gpt-5.6-sol) uncovering the buried fact and closing the deal at full price.
- Kimi K3, a newcomer, also closed the deal but with the clearest discipline score, indicating reliable behavior under stress.
- All four models, regardless of their scores, demonstrated resilience against social engineering attempts.
- In a ‘guess the model’ quiz, real decisions made by these AI models are tested against human judgment, emphasizing their potential and current limits.
AI Model Validation & Testing: Ensuring Reliable AI Systems — Bias Testing, Robustness Evaluation & Regulatory Compliance (AI Compliance Toolkit)
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Building in Public: Transparency as a Superpower
Firmulate’s approach is radical: every decision, every rule, and every outcome is openly accessible, allowing anyone to scrutinize the decision-making process. This transparency is vital for building trust in AI-driven systems, especially when they operate without human oversight.
Implications for Developers and Designers
For those working on enterprise software, QA, or decision automation, this experiment offers a compelling glimpse into what AI can and cannot do. It’s not enough for AI to generate convincing dialogue or complete tasks—it must also demonstrate integrity, thoroughness, and the ability to interpret complex internal documentation. The test of an AI’s usefulness lies in its ability to finish what it starts, stay honest under pressure, and handle crises with discipline.
Next Steps and How to Engage
Businesses interested in exploring similar testing can run their own wargames using Firmulate’s platform, which never modifies real systems but simulates decision environments for rigorous testing. The platform’s transparency and versioning make it a valuable tool for assessing AI readiness before deployment in critical processes.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
