Sten Vesterli's Blog

Making IT Live up to its Promise

Tag: Competence

Criminally Bad Project Management

IT Leadership, ,

Sometimes, failed IT projects cost real money. Like it just did for British bank TSB, who was fined about $60 million for their shambolic IT migration. The disaster locked people out of their accounts for weeks, and the total cost to the bank is now approaching $500 million with payments to customers, project post-mortems and IT cleanup.

“The firm failed to plan for the IT migration properly, the governance of the project was insufficiently robust and the firm failed to take reasonable care to organise and control its affairs responsibly and effectively, with adequate risk management systems,” the report from the banking authority concluded.

Those words don’t apply to any of your IT project, do they?

Very Few Things are Impossible

IT Leadership, , , ,

You can get anything you want. But you have to ask for it. The U.S. had an election last Tuesday, and they still don’t know the result. We had an election in Denmark on Nov 1st. As usual, our result was ready before midnight on election day.

Now, we actually count paper ballots here in Denmark, and the U.S. uses computers. But that isn’t the whole explanation of why we are so much faster. The main reason is that we have decided that we want a quick result. Thus, we have a cut-off date for advance voting three days before election day, and all the advance votes are ready to count on election day. Vote counting is easily parallelized, and we have enough people counting. The U.S. could do the same, but they have prioritized other factors.

When IT says something can’t be done, it is rarely true. It might indeed be difficult, or expensive, or require you to give up functionality of higher value. If you work in IT, don’t say something is impossible. If you request work from IT, don’t accept the answer of “impossible.”

It’s Expensive to Try to Get By With the Cheapest Resources

IT Leadership, , , , , ,

Talent is expensive. Not paying for talent is more expensive. Microsoft gets that. The U.S. Department of Defence doesn’t.

The Microsoft bug hunting program has a maximum payout of $250,000, and they did pay out $200,000 this year. You would think a crucial national defence vulnerability would merit a bigger bounty that finding a flaw in the Microsoft hypervisor, wouldn’t you? The DoD pays out $500 for a high-severity bug, and a whopping $1,000 for a critical issue.

Your developers are rewarded for shipping functionality. They don’t have the mindset to find the vulnerabilities. To build secure systems, you need to offer a bug bounty, or hire outside experts to do security review, or create your own internal white-hat hacker team. It does cost money. But security breaches cost much more.

The Tolstoy Principle in Action

IT Leadership, Technology That Fits, , ,

This is what failure looks like: 50% one-star reviews. The other half is five-star reviews. Assuming these are not all from the app developers themselves, the app apparently can work. It just didn’t work for me, nor for many others.

I call this the Tolstoy principle: All successful apps are alike; each unsuccessful app is unsuccessful in its own way. The end-user does not care that 98% of your back-end infrastructure is running. They care that they can complete their task. And if one critical component fails, your app is a failure. Like this one from my local supermarket chain.

When you build systems, is all the attention lavished on a cool front-end app? Unsexy back-end services are equally important.

Unnecessary Complexity

IT Leadership, Technology That Fits,

Why use a proper screwdriver when you have a multi-tool? It is true that it is a lousy screwdriver, but it can do a dozen other things. That’s the thinking behind using Microsoft Windows for Point-of-Sale terminals. It turns out to be a bad idea. It can take up to 40 minutes for a Windows 11 machine to install the latest update, and in the meantime you are unable to do business.

The problem is not throwing an overpowered machine at the task. A Raspberry Pi works fine for a home weather station even though it is only using 0.01% of its capacity. The problem is adding unnecessary complexity. A Windows 11 workstation is running literally hundreds of services, 98% of which are not necessary for Point-of-Sales functionality. The more components you have, the more potential problems you will have, and the harder it will be to find them when they occur.

You would never allow your IT architects to use over-complicated components with dozens of unnecessary interactions, would you?

Stop Speaking While You Think

Spiritual Programmer, ,

Do you, like, use too many, like, filler words? Sitting in a coffee shop in New York a few weeks back, I noticed that every fifth word in New York English is “like.”

Research shows that people who use more filler words are considered less intelligent, and their arguments are considered weaker. So it’s a good idea to get rid of your filler words. Here is an exercise: Record yourself on your phone speaking on any topic for one minute. Listen to the recording and count how many filler words you use per minute. Like, uh, ah, um, er, well. If you use more than one, you should improve.

Now record yourself again for one minute, this time making an effort to simply say nothing when you need to think. Pause and think instead of just babbling a filler word. People who pause while speaking are considered more intelligent. Listen to your recording and count your filler words. Hopefully, you have fewer. Also, notice that a pause of one second – the time normally filled with a useless word – is not a problem at all for the listener.

What Happens Then?

IT Leadership, Technology That Fits, ,

There is an easy way to avoid making stupid decisions: Asking “what happens then?” A decision is exposed as stupid when it turns out that the decision-maker did not carefully think through the consequences. Bad decisions occur when someone only looks at the immediate result.

New York City dodged a bullet when they started implementing bike lanes in the narrow streets of Manhattan. They could easily have made the stupid decision of simply marking a part of the street as a bike lane. Fortunately, someone clever at City Hall asked herself: What happens then? If you had simply painted bike lanes on streets, thoughtless New Yorkers would have wiped out bicyclists by the thousands with their car doors. So New York decided to paint a separation area between the car parking area and the bike lane. Clever.

Next time you are faced with a decision, try asking “what happens then?” several times. You might find this saves you from doing something stupid.

Are you Monitoring Important Systems?

IT Leadership, ,

New York is replacing their payphones with LinkNYC access points providing free calls, 911 calls, free WiFi, charging, and more. You would think such a system would warrant professional monitoring. Nevertheless, some of these devices just show a blue screen of error messages followed by a Linux login prompt.

  • Monitoring of crucial systems must include an automated mitigation action and reporting to a 24/7 operations center.
  • Monitoring of important systems needs immediate alerting to staff on call.
  • Monitoring of normal systems only needs to log a trouble ticket to be addressed by regular staff during working hours.
  • Low-priority systems do not need active monitoring.

It seems these kiosks are not as important to the company running the system as they were to the Mayor promising them.

Does every system on your central system list have a monitoring priority? When was the last time you checked with the person with the technical responsibility what monitoring is in place?

Engineering a Crisis

IT Leadership, Technology That Fits, ,

After imposing a loss of several hundred million dollars on airlines and annoying millions of passengers, the FAA has now stopped its publicity stunt. 90% of U.S. aircraft are now cleared to perform instrument landings even at airports near 5G towers.

They could have done this any time in the two years since the 5G licenses were awarded. However, quietly doing their job was not on the FAA’s agenda. After their failures led to hundreds of deaths in the Boeing 737-MAX8 disasters, they wanted to prove that they now take their job insuring safety seriously. They, therefore, engineered a crisis that put them on the front pages of newspapers nationwide before eventually doing what they should have done more than a year ago.

Don’t let corporate image considerations lead you to fail your customers. In short, don’t be like the FAA.

Public Incompetence

IT Leadership,

California shows why you don’t want to entrust your government with more data than it absolutely needs. California is building a database with all donations to political and non-profit organizations. Free-speech advocates from a Charles Koch foundation to the NAACP are against it, and the Supreme Court will soon rule whether this is okay. As a European, I don’t much care either way.

The IT security part is interesting, though. It is fairly sensitive information whether someone dontated to Donald Trump or Black Lives Matter. California of course promises to protect it, but they have built a website where you can simply change the URL to get access to every donation record in the entire database.

That is mind-boggling incompetence. I am not blaming the individual developer, though an experienced lead programmer could have spotted this glaring vulnerability. But I am blaming the IT leaders in the organization that have failed to put any kind of security review in place, even for highly sensitive data. That is a firing offense in my book. Maybe a competent CIO should run against Governor Newsom in the California recall election.