Who Thinks About Risk?

A “Silicon Valley Bank Risk Management Department” T-shirt is the latest in ironic workwear. Not that SVB seems to have much risk management – their Chief Risk Officer stepped down in April last year, and the position was vacant for eight months.

Does anybody have the Risk Manager position in your IT organization? Every project creates a risk matrix and mitigates the worst risks, but once the project is complete, risk management evaporates in many organizations. The CISO does some risk management, but many IT risks are outside her remit. And risk management falls squarely in the “important, not urgent” category that always gets pushed to the back of the task list…

Beware of Asymmetric Risk/Reward Profiles

Would you continue to sell a lock based on technology that has been known for 14 years to be trivially easy to hack? Of course not! But Scantron in Denmark has merrily been foisting insecure locks on unsuspecting Danish apartment administrators. Even after a worried renter told them about the problem in several emails and even physical letters (!), they ignored the problem. It took a media shitstorm to make them realize the errors of their ways.

Digital locks have an asymmetric risk/reward profile. The reward is small – you save a little by not having to administer physical keys and re-key locks. The risk is huge – someone might copy a key, turn it into a master key, and rob hundreds of apartments.

When you are evaluating digitalization projects, be very careful about those with such an asymmetric profile. Almost every organization has digitalization projects with a better risk/reward balance than digital locks…

Good Intentions are not Enough

“We have the ambition to test disaster recovery twice a year.” That’s not something anybody in a professional IT organization would say, is it? Ambition? I have the ambition to create a spam- and hate-speech-free Twitter alternative powered by unicorns and rainbows, but unless I act on my ambition, nothing will happen.

Nevertheless, critical Danish infrastructure was operated on that principle. The common login system that everything from banks to tax authorities to municipalities uses is operated by a company called Nets. They apparently got to write their contract with the state themselves because it contains the ridiculous “ambition” instead of an actual requirement.

They did run a test on May 28, 2020. They did not run a test in November 2020, as was their ambition. Nor in May or November 2021. Not even in May 2022 did they test it. So when they crashed the system in June 2022 due to undocumented changes and other unprofessional shenanigans, the disaster recovery unsurprisingly failed.

Please tell everyone this story. When you are done laughing at the incompetence of central Danish authorities and their vendors, make sure you are testing your own disaster recovery…

It’s Expensive to Try to Get By With the Cheapest Resources

Talent is expensive. Not paying for talent is more expensive. Microsoft gets that. The U.S. Department of Defence doesn’t.

The Microsoft bug hunting program has a maximum payout of $250,000, and they did pay out $200,000 this year. You would think a crucial national defence vulnerability would merit a bigger bounty that finding a flaw in the Microsoft hypervisor, wouldn’t you? The DoD pays out $500 for a high-severity bug, and a whopping $1,000 for a critical issue.

Your developers are rewarded for shipping functionality. They don’t have the mindset to find the vulnerabilities. To build secure systems, you need to offer a bug bounty, or hire outside experts to do security review, or create your own internal white-hat hacker team. It does cost money. But security breaches cost much more.

Why Projects Without Business Cases are Shot Down

I just had a customer attempt to start a project without a business case. Such projects are usually driven by the desire to use a specific technology and with a vague idea that this would somehow benefit the end user.

If the IT department is strong, some of these orphan projects get started. They might be successful. However, since the organization has no idea of the business benefit, it is blind luck if the benefits exceed the cost.

If the business prevention department (compliance/legal) is strongest, they are shot down. There is always a reason not to make any changes. A project without a business case can be mortally wounded by any objections about compliance, GDPR, security, etc.

That is why every project needs a business case. It prevents IT from wasting money on something that will not add value, and it prevents compliance & legal from killing projects with a positive business impact.

Do your projects have solid business cases? If not, get in touch, and I’ll help you.

Who is in Charge of Outside-the-Box Thinking?

Everyone can track your license plate – not just the cops. A Belgian security researcher noticed that most parking apps do not validate that you actually own the license plate you add to your app. That means a stalker can add his victim’s license plate to his app and immediately be notified whenever that person parks anywhere…

This is another example of the inside-the-box thinking that developers are prone to. The developers of the Kryptonite bike lock had made it out of extra reinforced steel. Too bad a weakness in the lock allowed a hacker to open it with half of a ballpoint pen.

Finding holes in a system is not just securing the login and checking the encryption. It involves examining the system and its environment and users. That is a skill most developers lack. You need a “red team” who can find the holes before you roll out something embarrassingly insecure.

Do You Know Where Your Data Is?

Facebook has no idea where they store your data. In a hearing, two senior Facebook employees admitted that they couldn’t say where user data was stored, much less ensure that it was all turned over to the authorities or deleted if required. The investigator said, “surely someone must have a diagram?” The engineers replied, “no, the code is its own documentation.”

The second law of thermodynamics applies to IT systems just like it applies to the rest of the world. It says that the amount of entropy, or disorder, inexorably increases unless someone spends energy actively trying to diminish it.

That becomes a problem when nobody spends time refactoring or cleaning up but lots of time adding new features, integrations, and dependencies. More than half of all organizations are where Facebook is: They don’t have and cannot establish the full picture of how their systems work. That places them at risk of catastrophic and irrecoverable failure. Can you establish a complete overview of your systems?

What is Your Time to Recover?

It wouldn’t take you three to four weeks to rebuild a critical system, would it? But that’s how they do things in the National Health Service in the UK. Doctors and hospitals have been advised that the central patient record system is offline due to a ransomware attack and will not be back until sometime in September. In the meantime, doctors will have no access to their patients’ medical histories and will have to keep notes on paper or in Microsoft Word on their laptops.

As a national health monopoly, the NHS will not be going out of business. But a private company that lost its manufacturing, logistics, or service management system for a month would be finished.

You do everything you can to prevent bad things from happening. But have you also planned contingent action in case something terrible does happen? The NHS hadn’t.

Documentation is Unnecessary Until You Need It

If you have a fire in your server room, your insurance pays out. Insurance is expensive, but a necessary part of your risk management strategy. For many risks, there is a way to get almost free insurance. Yet few people take it. I am talking about documentation.

A chocolate factory in Belgium didn’t follow its own processes and did not document its production. When kids started falling sick with salmonella all over Europe, suspicion fell on the Kinder egg factory in Arlon. The authorities asked for the production documentation. Because the factory couldn’t provide it, the whole plant was shut down. If they had had documentation, they would have been insured against this risk. They could have shut down just one production line instead of the whole plant.

So the reason you might not be able to get chocolate eggs this Easter is bad documentation.