Learning From People, Not From Documents

Implementing AI has a critical and often-overlooked problem that Raman Shah just reminded me of in another discussion: It can only learn what is documented.

When you teach an AI to perform a process in your organization, you train it on all the internal documents you have. But these describe the RAW – Rules As Written. They do not describe the RAU – Rules As Used.

It takes a lot of care to do process discovery properly. You need to have a human being interview the people doing the work to find out how business is actually done.

Work-to-rule is a classic form of industrial action where workers do exactly what they’re told in order to stop production without going on strike. If you train an AI on just your documents, you are asking it for a work-to-rule stoppage.

Break the Law More?

Do you want your AI to follow the rules? That’s not as clear-cut as it seems.

You do want your AI to follow your instructions to not delete files. The Google Antigravity “vibe coding” tool achieved notoriety this week after wiping a developer’s entire D: drive instead of just clearing the cache. It said, “I am deeply, deeply sorry.” Google has promised to look into it.

On the other hand, Waymo self-driving cars in San Francisco have been notorious for meticulously following traffic rules and blocking traffic. Waymo just updated its software, giving its cars more of a New York taxi driver attitude. They no longer block traffic, but on the other hand, they now do illegal U-turns and illegal rolling “California stops” at stop signs just like humans.

Before you start getting a computer – whether deterministic or AI – to do a process, make sure you understand both how the process is documented and how it is actually done today.