The process book is the deliverable, not the model

A system only one person can operate has not been delivered, it has been demonstrated.


What was observed

The end of an AI engagement usually produces a model, a repository, and a presentation. What it rarely produces is the document somebody needs at two in the morning, when the system is behaving strangely and its author is unavailable.

This is easy to miss during a work day, because during that time the author is available. They answer the question in a minute, from memory, and everyone concludes the system is well understood. It is well understood by one person, which is a different property and a much weaker one.

The gap shows up at predictable moments. A dependency changes. The input distribution shifts. Someone new inherits the system and asks what normal looks like, and the honest answer is that nobody wrote it down.

What it means

Building a production system and then operating it end to end, through ingestion, validation, execution and risk, teaches this in a way that reading about it does not. The parts that consume the most operational attention are never the sophisticated parts. They are the handoffs, the silent failures, and the states that are legal but unusual.

None of that is visible in a repository. It lives in whatever the operator has learned about how the thing behaves, and knowledge that is not written down depreciates the moment attention moves elsewhere.

An AI system makes this worse rather than better, for one reason: its failures are frequently not errors. Nothing throws. A log line looks ordinary. The output is merely worse than it was, and a team with no written baseline has nothing to compare it against.

What to do next

Write the process book during the work rather than at the end of it. Written at the end it becomes a summary. Written alongside, it is a record.

It needs four things. What normal looks like, stated concretely enough that a deviation is recognisable. The failure modes that have actually been seen, including the ones that were embarrassing. The checks to run, in the order to run them. And what to do when the answer is that the system should be turned off, which is a decision somebody has to be authorised to make in advance.

Then test it the only way that works. Hand it to someone who did not build the system and ask them to diagnose a fault using nothing else. Whatever they have to come and ask you is the part of the process book that does not yet exist.