Building Resilient AI-powered Software, And Planning For When Things Go Wrong
AIThis post was created with the assistance of artificial intelligence (AI).

TL;DR

Buying for a business?Offer from Amazon

Get business pricing on monitors, keyboards and dev gear

  • Business-only prices and quantity discounts
  • Tax-exempt purchasing
  • Multiple users, one account, clear invoices
As an affiliate, we earn on qualifying purchases.

The Pragmatic Engineer published excerpts from Sam Newman’s 2026 book, Building Resilient Distributed Systems, covering disaster recovery and resilience for AI-powered software. The material outlines recovery targets and patterns, and discusses risks including changing models, provider dependence and limited computing capacity.

The Pragmatic Engineer has published excerpts from Sam Newman’s new book, Building Resilient Distributed Systems, examining how software teams can prepare for catastrophic outages and manage risks in AI-powered systems. The report focuses on disaster recovery planning, infrastructure constraints and the challenges of relying on changing AI models and providers.

In an introduction to the excerpts, The Pragmatic Engineer says Newman spent three years writing the book, one year longer than originally planned, and reworked its material to account for AI. The newsletter says the published selection draws on two chapters, but covers less than a full chapter of the book’s 16 chapters.

The disaster recovery chapter frames recovery as a balance among cost, complexity and acceptable downtime or data loss. Newman describes two targets for setting requirements: the recovery point objective (RPO), which sets the maximum acceptable period of lost data, and the recovery time objective (RTO), which sets how long a service can remain unavailable. An hourly backup, for example, could leave up to an hour of data at risk, giving the system an RPO of one hour.

The excerpts also list recovery approaches ranging from backup-and-restore to pilot-light and warm failover, as well as multisite active-active designs. The report’s AI-focused topics include model-provider resilience, changes to models and observability. It also says the book addresses uncertainty through options such as using multiple models or vendors, making costs transparent and reviewing vendors regularly.

At a glance
reportWhen: Published in 2026; the report describes…
The developmentThe Pragmatic Engineer published excerpts from Sam Newman’s newly released book on disaster recovery and resilient AI-powered software.

Recovery Targets Shape Outage Plans

The guidance matters because a disaster recovery plan is not simply a promise to restore service: it involves explicit choices about how much data may be lost and how long users may wait. RPO and RTO make those choices measurable, allowing teams to compare a recovery design with the needs of the service and the cost of maintaining it.

For AI-powered products, reliability also depends on components outside a team’s direct control. Newman’s excerpt points to provider availability, model changes and computing capacity as planning concerns. The newsletter specifically notes ongoing GPU and CPU shortages, arguing that teams should understand vendor constraints and prioritize critical services rather than assume capacity will always be available.

Amazon

disaster recovery backup software

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

From Site Outages to AI Dependencies

The book’s disaster recovery chapter considers failures that can take down a whole site, including fires, floods, electrical problems and failures in underlying cloud infrastructure. Newman uses “site” as a broad term for a cloud region, data center or comparable place where computing resources run, so the recovery patterns can apply across public-cloud and on-premises environments.

The newsletter presents the excerpts as advice for engineers responsible for reliable systems, not as a report of a specific outage or new industry-wide measurement. It says the book also discusses the challenges of using multiple cloud vendors as part of disaster recovery, but the supplied excerpt does not provide those details.

“The recovery point objective (RPO) defines the maximum amount of time that can have passed since the data was last backed up or synchronized.”

— Sam Newman, in an excerpt from Building Resilient Distributed Systems

Amazon

AI model monitoring tools

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Recovery Advice, Not Outage Data

The source is an editorial introduction and book excerpt, not an independent study measuring how often AI-related outages occur or whether AI coding tools have increased incident rates. The introduction says more code deployments and bugs are contributing to more production incidents, but provides no figures, time period or comparison baseline for that claim.

The supplied material also does not specify Newman’s detailed recommendations for model monitoring, multi-provider operation or vendor reviews. It does not give cost estimates or recovery benchmarks for the different disaster recovery patterns, so readers cannot compare their performance from this excerpt alone.

Amazon

distributed system resilience hardware

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Apply Targets to System Needs

The next step for teams using the guidance is to set acceptable data-loss and downtime targets, then choose a recovery pattern that fits their service requirements and resources. That planning also calls for checking provider and infrastructure limits, especially where services depend on scarce GPU or CPU capacity.

The Pragmatic Engineer says the excerpts cover only a portion of Newman’s book, which is published by O’Reilly Media. Further details on vendor strategy and the other topics would depend on the remaining material; the source does not announce a follow-up report or a specific publication event beyond the book’s release.

Amazon

cloud infrastructure failover solutions

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Key Questions

What is the news?

The Pragmatic Engineer published excerpts from Sam Newman’s 2026 book, Building Resilient Distributed Systems, on disaster recovery and AI software resilience.

What do RPO and RTO mean?

RPO is the maximum acceptable period of potential data loss. RTO is the maximum acceptable time a service remains unavailable after an incident.

Which recovery approaches are mentioned?

The report lists backup-and-restore, pilot-light failover, warm failover and multisite active-active as approaches discussed in the book.

It points to dependence on model providers, models changing over time, observability needs and possible constraints on computing infrastructure. The excerpt does not quantify these risks.

Does the report show that AI coding tools have increased outages?

No. The introduction says increased code deployments and bugs are contributing to more production incidents, but it provides no supporting statistics or comparison baseline in the supplied material.

Source: rss

HALLOWEEN

Halloween Picks

As an affiliate, we earn on qualifying purchases.

You May Also Like

IBM Expands AI And Quantum Research With IIT Bombay And IISc

IBM is extending research partnerships with IIT Bombay and IISc in Indic-language AI, agentic systems, energy models and quantum-HPC algorithms.

CodePen 2.0

CodePen has announced the release of Version 2.0, introducing new features and a redesigned interface aimed at improving user experience and collaboration.

Rocket Lab Surges In Global Coverage

Rocket Lab has experienced a surge in international media mentions, indicating increased global interest and visibility in its space launch services.

Asynchronous I/O In DuckDB: Work, Thread, Work

DuckDB releases new asynchronous I/O capabilities using a work, thread, work model, enhancing query performance and concurrency.