How to Avoid Techno-Blindness

Techno-blindness is a dangerous affliction. It is a disease of over-optimism mainly affecting people in the technology industry. The symptom is overconfidence that a system works as intended and a lack of awareness of what might go wrong.

Somebody in Moscow thought it was a cool idea to have a computer play chess with children, using a robotic arm to move the pieces. Until a child made an unexpected movement. The robot grabbed his hand and broke his finger. TuSimple is building autonomous trucks, and one of them accidentally executed an old instruction, causing it to turn left in the middle of the highway. Fortunately, nobody was injured as the truck veered across the I-10 and slammed into a barrier.

Important systems need independent safeguards. That means a completely separate piece of code that can intervene if the output of an algorithm lies outside some boundary. A truck shouldn’t be able to turn left at high speed. A robotic arm shouldn’t move on the chessboard until the player’s hands are off the pieces.

As a CTO, it is your job to ensure there are safeguards around important systems. You cannot depend on techno-blind developers to do this by themselves.

Irrational Optimism

Beneficial Intelligence is out. This week: Irrational optimism. IT people are too optimistic about schedules and business cases. It is a natural consequence of our ability to build something from nothing. As a CIO or CTO, you need to make sure you have some pragmatic pessimists to point out the things that might go wrong. Because IT is too optimistic and Legal & Compliance is too cautious, IT organizations often turn to outsiders like me.

Listen here or find “Beneficial Intelligence” wherever you get your podcasts.