Once you Grow up, you Need to Stop Moving Fast and Breaking Things

Moving fast and breaking things can be fine for a startup. They might need to iterate several times and maybe even pivot once or twice before they achieve product/market fit. It is not OK for an established business. Facebook has long since given up on this strategy, but Twitter, under Elon Musk, has rediscovered it. By thrashing around and changing direction daily, they are alienating both the users and the advertisers who were supposed to pay. If you want to move fast, roll out changes to a small percentage of your users. A mature continuous delivery organization practices blue/green deployment, but even if you are not doing CI/CD, you can still test changes with a small subset of your users. Don’t uncritically inflict the latest great idea on your entire user population. #itleadership #innovation #makeitliveuptoitspromise

Beware of Asymmetric Risk/Reward Profiles

Would you continue to sell a lock based on technology that has been known for 14 years to be trivially easy to hack? Of course not! But Scantron in Denmark has merrily been foisting insecure locks on unsuspecting Danish apartment administrators. Even after a worried renter told them about the problem in several emails and even physical letters (!), they ignored the problem. It took a media shitstorm to make them realize the errors of their ways.

Digital locks have an asymmetric risk/reward profile. The reward is small – you save a little by not having to administer physical keys and re-key locks. The risk is huge – someone might copy a key, turn it into a master key, and rob hundreds of apartments.

When you are evaluating digitalization projects, be very careful about those with such an asymmetric profile. Almost every organization has digitalization projects with a better risk/reward balance than digital locks…

IT Leadership has to Harness the Power of AI

AI has finally gotten really useful inside the IT organization. Most of the examples on the internet are frivolous and amusing, like how to remove a peanut butter sandwich from a VCR, written in the style of the King James Bible. But ChatGPT is helpful for mundane tasks in IT as well.

I’ve been fixing open issues in a small open-source project recently. One of the issues was that part of the code would concatenate strings to build SQL statements. That’s a classic SQL Injection vulnerability. ChatGPT can fix these bugs faster than I can. So I tell the AI, “please rewrite the following to use bind variables,” and give it the code.

Another example is working we legacy shell scripts. My sed/awk skills are rusty, but I can give a convoluted shell statement to ChatGPT, and it will patiently explain all the options and exactly how it works.

Many of your programmers are already playing with ChatGPT, GitHub Copilot, and other AI tools. You might as well embrace it. Set up a knowledge-sharing community for those curious about how AI can help IT. Have them present to you and the rest of the IT department. You’ll be amazed if you haven’t played with ChatGPT and its ilk.

Blockchain is Still a Solution Looking for a Problem

It turns out nobody wanted a blockchain solution. There are still crypto enthusiasts hodling their Bitcoin, but enterprise blockchain was a solution in search of a problem.

I did believe Danish shipping giant Maersk Lines and IBM had found a place where it made sense to build something blockchain-based when they announced their TradeLens platform. The idea was that all the many, many people involved in shipping a container of plastic bric-a-brac from Shenzen to Long Beach would all put their information on a blockchain. That would provide an immutable history of everything about that container.

After IBM closed down its entire blockchain business earlier this year, it was a matter of time before Maersk pulled the plug. Today, they admitted that “TradeLens did not reach commercial viability,” and the project is officially dead.

I believe a land register in a corrupt country somewhere was also planning to use blockchain, but it’s been a while since I last heard about it. In all likelihood, the existing corrupt businessmen and politicians have killed it.

If you know of any successful enterprise blockchain project, I would love to hear about it.

You Don’t Have to Move Just Because You’re Ready

I was worried when I saw Denmark ranked no. 4 in “The Global Cloud Ecosystem Index 2022.” I was afraid that we had somehow stumbled into the cloud trap without my noticing. But it turns out the index is not about actual cloud adoption, only cloud readiness.

Being ready for the cloud means having affordable, fast internet connections, digital public services, data protection regulations, and a well-educated workforce. I’m all for that.

But the fact that we can doesn’t mean we should. Just like the fact that you could move some of your services to the cloud is not an argument for doing it. There are some systems where there is a sound business case for moving to the cloud. But for most existing systems, attempting to move to the cloud destroys value.

Good Intentions are not Enough

“We have the ambition to test disaster recovery twice a year.” That’s not something anybody in a professional IT organization would say, is it? Ambition? I have the ambition to create a spam- and hate-speech-free Twitter alternative powered by unicorns and rainbows, but unless I act on my ambition, nothing will happen.

Nevertheless, critical Danish infrastructure was operated on that principle. The common login system that everything from banks to tax authorities to municipalities uses is operated by a company called Nets. They apparently got to write their contract with the state themselves because it contains the ridiculous “ambition” instead of an actual requirement.

They did run a test on May 28, 2020. They did not run a test in November 2020, as was their ambition. Nor in May or November 2021. Not even in May 2022 did they test it. So when they crashed the system in June 2022 due to undocumented changes and other unprofessional shenanigans, the disaster recovery unsurprisingly failed.

Please tell everyone this story. When you are done laughing at the incompetence of central Danish authorities and their vendors, make sure you are testing your own disaster recovery…

This 47-Year-Old Classic Will Improve Your IT Skills

There are two kinds of people in the world: Those who know who Fred Brooks was and those who don’t. If you are an IT professional in the second group, you can step up your game dramatically by reading his seminal book “The Mythical Man-Month.”

Fred Brooks managed IBM System/360, the project that produced the first real general-purpose computer back in the 1960s. He distilled his experience from this 5,000-man-year project into the first edition of TMMM in 1975 and the expanded anniversary edition from 1995 stands on my bookshelf. When I meet other experienced IT architects, as at Software Architecture Open Space in Copenhagen this month, people will use phrases like “second-system effect” that originated with Brooks. He passed away yesterday after a long and productive life full of accolades.

To commemorate Fred Brooks, I’m inviting you to join a series of online discussions on IT best practices and what we can still learn from The Mythical Man-Month. We’ll meet on Zoom every Thursday at 5 pm CET = 11 am EST = 8 am PST. We’ll discuss one chapter from the book and how it applies to our work in IT today. I expect each meeting will be 30-60 minutes, and we’ll record it for those who can’t make it. We start next Thursday, November 24. Sign up here: https://vester.li/tmmm

Using the Power of UX for Good or Bad

You can easily manipulate users. Using design tricks to confuse and deceive users is known as “Dark UX,” and Airbnb has been an enthusiastic practitioner. For example, American users have always been surprised that their great deals look much less great after humongous compulsory “cleaning fees” are added at the last step.

I never saw this trick in Denmark because such shenanigans are illegal here. Airbnb power users know to search for Airbnb rentals in the US on the Australian site because deceptive practices are also illegal there.

Under pressure from users and regulators, Airbnb has stalled for years, implausibly claiming technical challenges in displaying the total price. However, it seems like the pressure has now grown too big to ignore, and even Americans should shortly be able to see the actual price.

User Experience knowledge is meant to help users, not trick them. You don’t want your company to become a byword for deception like Airbnb has become.

Denmark is Dangerously Unprepared – Are You?

Denmark is not prepared for IT disasters and attacks. The state auditors have chosen 13 out of the approx. 4,200 public IT systems and looked at their recovery plans and procedures. A few were fairly well prepared, most were not, and one system was completely unprepared for anything to go wrong.

None of the recovery plans were adequately tested, and five systems had not tested their recovery plan at all in the last three years. For outsourced systems, half of the contracts did not require testing the recovery plan (!).

But at least the Danish state has an office that examines these things and issues a report. Who is responsible for evaluating the disaster recovery plans for critical systems in your organization? You cannot leave that to the individual system owners.

Handing Off Your Problems to Someone Else

Today is the day when up to 300,000 Danes can no longer access their online banking. They also cannot use any of the gazillion public services that require a login. That’s because the old public ID system in Denmark has been retired, and everyone has to use the insecure and shoddily built new one.

The reason thousands of people are left behind is the cumbersome signup process that – among other things – involves scanning the chip in your passport with a modern smartphone. It turns out many people can’t figure out how to do that. But that is not a problem for the organization behind the ID system. They simply tell users to show up at the local service point in their town for help.

It is, however, a problem for the overworked local service center employees. They are staffed to (barely) manage their usual work. Dumping 500,000 IT support tasks on them has predictably led to huge waiting times for an appointment for anything.

Don’t allow your IT systems to dump their problem somewhere else and declare themselves a success.