Denmark is Dangerously Unprepared – Are You?

Denmark is not prepared for IT disasters and attacks. The state auditors have chosen 13 out of the approx. 4,200 public IT systems and looked at their recovery plans and procedures. A few were fairly well prepared, most were not, and one system was completely unprepared for anything to go wrong.

None of the recovery plans were adequately tested, and five systems had not tested their recovery plan at all in the last three years. For outsourced systems, half of the contracts did not require testing the recovery plan (!).

But at least the Danish state has an office that examines these things and issues a report. Who is responsible for evaluating the disaster recovery plans for critical systems in your organization? You cannot leave that to the individual system owners.

How Do You Make Sure You Keep Up?

Did you learn anything this week? Every industry is changing rapidly, and the IT industry more than most. Those who keep their noses to the grindstone every day will miss important trends. There are new technologies, new tools, and new ways of working.

I was discussing the future of IT with some of the sharpest minds in Denmark at the Software Architecture Open Space in Copenhagen yesterday and came away with new insights and provocative rebuttals to some of my entrenched notions about how organizations can be successful with IT.

If you are in a leadership position in IT, how do you ensure that your key players take time out from their day-to-day tasks to learn what is happening in the industry?

Handing Off Your Problems to Someone Else

Today is the day when up to 300,000 Danes can no longer access their online banking. They also cannot use any of the gazillion public services that require a login. That’s because the old public ID system in Denmark has been retired, and everyone has to use the insecure and shoddily built new one.

The reason thousands of people are left behind is the cumbersome signup process that – among other things – involves scanning the chip in your passport with a modern smartphone. It turns out many people can’t figure out how to do that. But that is not a problem for the organization behind the ID system. They simply tell users to show up at the local service point in their town for help.

It is, however, a problem for the overworked local service center employees. They are staffed to (barely) manage their usual work. Dumping 500,000 IT support tasks on them has predictably led to huge waiting times for an appointment for anything.

Don’t allow your IT systems to dump their problem somewhere else and declare themselves a success.

Are You Sure Your Backup System Works?

Why did all the trains in Denmark stop on Saturday? Russian hackers may or may not have been involved, but Danish incompetence was.

The Danish State Railways (DSB) has digitized all the paper that a train driver used to carry. That’s temporary speed restrictions, track works, and deviations from the standard schedule. They have also outsourced their digital solution to an amateurish vendor, and neither the vendor nor DSB had a backup solution. So when the vendor shut down the system due to an unspecified “security issue,” the trains stood still.

I’ve boarded a Delta Airlines flight with a hand-written boarding card on a scrap of paper. A professional organization continues to run, though slower, without its computers. An unprofessional organization like DSB is paralyzed. Are you like Delta Airlines or like the Danish State Railways?

Have You Earned the Right to do Away With Estimates?

Trust is the crucial precondition for healthy cooperation between IT and the business. I’ve been discussing #NoEstimates quite a bit with various people since my post yesterday, and everywhere this approach works, there is high trust. This is something that IT builds over the years by delivering as promised. When you have that trust, you can spend less time on estimates.

A Hollywood instructor managed to create quite advanced and artful movies inside one of the major commercial studios. He was asked how he got away with it when every other instructor at that studio was grinding out bland standard fare. He explained that his secret was to start shooting at 8 am sharp every morning. When the producer showed up mid-morning, he could see that the first two scenes were already done and dusted, so he left the instructor alone to do his thing.

The origin story of #NoEstimates on Woody Zuill’s blog is very similar: The team started by proving they could provide the most desired value quickly, thus creating a seed of trust. But unfortunately, in many organizations, IT has a track record of over-promising and under-delivering, built up over many years. That’s why the business demands detailed estimates.

To do away with estimates, you first need to build trust.

Why Should the Business Trust You With Their Money?

“Give us a bag of money and go away.” That seems to be the thinking of most in the #NoEstimates movement. They have, of course, misunderstood the original concept, just like people who claim to do Agile when all they’ve done is to do away with the documentation. I agree that estimation is hard and software is complex, but asking the business to commit money for unknown benefits in the uncertain future represents monumental hubris. The real world works by comparing costs and benefits, even though both cannot be evaluated exactly.

I’ll be meeting some of the best and brightest IT architects in Denmark at the annual Software Architecture Open Space next week. This is an open-format conference, and I noticed some of the other participants have already brought up estimation and #NoEstimates as a topic. I’m looking forward to an interesting discussion. If you are in the vicinity of Copenhagen on Nov 3rd, I encourage you to participate in SAOS as well. You’ll surely learn something.

The Problem is the Humans, Not the Technology

The weakest link is the human. Microsoft does keep the software in their Azure cloud up to date with the latest patches but still managed to lose 2.4 terabytes of data belonging to 65,000 customers in 111 countries. The reason is that someone at Microsoft misconfigured a storage container.

This story became public because a security company wanting to sell its scanning solution posted it. They also informed Microsoft, who quickly secured the container. But for every white-hat hacker scanning the internet for unsecured storage, there are ten black-hat hackers siphoning off your secrets and selling them.

By buying a high-level cloud service from a reputable vendor, you can be sure that it runs on well-patched servers without known vulnerabilities. But you’ll have no idea when your cloud vendor fails to secure some lower-level service until you read about it in the news.

There are Many Reasons Not to Move to the Cloud

You don’t save anything by moving to the cloud. Ask around – how many of the organizations you know who moved to the cloud have reduced operations headcount? Some things are simpler in the cloud, but many others are more complicated.

You enforce some good security practices because there is no way to NOT install the latest security patches. And you can quickly spin up an extra testing environment.

But unless you really have a highly variable load, or you are starting something new where you don’t have a clue how much power you’ll need, the cheapest option is to buy some hardware and put it in your server room.

The next time one of the vendors tells you how much you save by moving to the cloud, take a really good look at the calculation. I’ll be happy to help you. You will likely find out that there isn’t a business case for moving.

Yet Another Project With No Business Case

There is nothing so good that you cannot do it badly. Case in point: Recycling. I’ve just received five new recycling containers in my summer cottage. The point is for me to sort plastic, cardboard, metal, paper, dangerous items, organic waste, and the remainder in separate compartments. I’m all for recycling, but I was curious about the business case for providing new plastic containers for summer cottages with limited amounts of waste and driving around with more big trucks on little dirt roads to collect the stuff.

It turns out there isn’t one. The danish Engineer’s Association has a weekly newspaper, and they have been running stories on this. Dispassionate calculation shows that the cost in money and CO2 of collecting and sorting several of these waste fractions far exceeds the benefit of recycling. For plastic waste, it turns out that we have to drive it – in trucks – to neighboring Germany because we don’t have a facility to reuse it in Denmark.

Surely, the government that invented this process would have a good counterpoint? Nope. I’ve been looking through several government websites. There is a lot of greenery and nice words, but no business case for recycling as much as we currently attempt to do.

So here in Denmark, recycling is a project with a worthy goal, political backing, and no business case. Have you ever seen something like that happen in IT?

Can You Trust Your Vendor?

Did you invite the hackers in yourself? Hundreds of German companies are waking up to the revelation that “German” cyber-security company Protelion is a front for a Russian company with links to Russian intelligence.

The hapless boss of the German IT Security agency even invited Potelion to sit on the German Cybersecurity Council. He is facing an unceremonious sacking…

Being able to roam freely inside the firewall and install agents with admin privileges is the dream of any hacker. There are at least one million devices running Protelion’s “security” software and the companies who invited Protelion in face a wholesale scrubbing of their entire IT infrastructure.

How do you plan to ensure that your security audits do not worsen your security?