When the Backup Is a Human Who No Longer Knows How to Do the Job
Capgemini ·
As organizations automate more work with AI, agents, and other intelligent systems, we spend a lot of time asking whether the technology is reliable. We spend less time asking what happens to the people who used to perform the work. The post When the Backup Is a Human Who No Longer Knows How to Do the Job appeared first on Capgemini .
Your AI system goes down.
No problem. The business continuity plan instructs you to switch to manual processes. But there’s an issue… nobody remembers how to do it.
As organizations automate more work with AI, agents, and other intelligent systems, we spend a lot of time asking whether the technology is reliable. We spend less time asking what happens to the people who used to perform the work.
A human fallback sounds reassuring, but a fallback only works if the human still has the skills to take over.
Automation changes people, too.
Think about a process that once required an employee to gather information, analyze it, decide, and then execute the next step. An AI agent may gradually take over much of that work.
At first, the employee supervises it closely. Then the agent becomes more reliable and the employee intervenes less often. Eventually, their role may shift from doing the work to monitoring the work.
That is efficient.
It can also change the skills people practice every day.
A 2025 study from Microsoft Research and Carnegie Mellon University looked at 319 knowledge workers using generative AI . The researchers found that higher confidence in AI was associated with less critical thinking. They also observed a shift from information gathering to verification, from problem-solving to integrating AI responses, and from task execution to task stewardship. The authors warn about the potential for long-term over-reliance and a reduced ability to solve problems independently.
However, it’s important to note that AI does not necessarily make people less capable, but skills that are rarely exercised can become harder to call upon when suddenly needed. And unfortunately, the moment we need them most may be during an incident.
We have seen this problem before, and it is not unique to AI.
Aviation has dealt with the tension between automation and human proficiency for decades. Guidance for pilots stresses that automation can improve safety and reduce workload, while also creating a risk of over-reliance and deterioration of manual skills if those skills are not practiced. The uncomfortable part is obvious: manual capability matters most when automation is unavailable.
Today, businesses may be creating a similar problem.
A continuity plan may say that if an AI agent fails, employees will temporarily perform the process manually. But when was the last time they actually did it? Is your manual fallback real?
Imagine an AI agent has processed a particular workflow for two years.
During that time, employees changed roles and new employees joined who were never trained on the original manual process. Procedures became outdated because nobody used them. The business process itself evolved around the capabilities of the agent.
Then the agent becomes unavailable.
Technically, you still have a human fallback, but operationally, you may not.
That means business continuity planning for AI should ask more than: can a human take over? It should ask:
Organizations routinely test backup systems, disaster recovery environments, and emergency procedures.
Human capability should be tested, too.
If a critical AI service has a manual fallback, occasionally exercise that fallback. Let people perform the process without AI. Check whether instructions still work. Identify knowledge that has disappeared and make sure enough people retain the skills needed to operate during a disruption.
The goal is not to resist automation, it’s to avoid building a continuity plan around a capability that no longer exists. Because “a human can do it” and “a human can still do it” are two very different statements.
Before putting manual process into your business continuity plan, make sure there is still someone who knows what that actually means.
The post When the Backup Is a Human Who No Longer Knows How to Do the Job appeared first on Capgemini .