Sten Vesterli's Blog

Making IT Live up to its Promise

Tag: Ethics

Don’t Use Illegal Defaults

IT Leadership, Technology That Fits,

You would never implement a system programmed to break the law, would you? The municipalities in Denmark did. If you get social security in Denmark, you are supposed to work at least 225 hours per year if you can. Those who can, and don’t, get less money. Those who cannot work are exempt from this deduction rule. The IT system has been programmed to automatically start reducing benefits unless a caseworker remembers to manually keep pushing the deduction date into the future. This means the municipalities save money by illegally reducing benefits for those citizens who do not have the energy to complain.

When you automate a process, your users will quickly come to accept the decision of the system. Make sure you have good defaults. At the very least, make sure they are in accordance with the law.

What Can and Cannot Be Said

IT Leadership,

Can you say “pay rise” in your company? At Amazon, that would not be possible. The internal social media app they plan to roll out for warehouse workers will filter out words like “union,” fairness,” and “plantation” (!)

Russian author Fyodor Dostoyevsky said, “The degree of civilization in a society can be judged by entering its prisons.” Similarly, the degree of civilization in an organization can be judged by its internal social media.

What does your internal communication platform and its rules say about your organization?

Pay attention to the rules

IT Leadership, Technology That Fits, , , , ,

It’s probably time to start paying attention to the rules. Inspired by the Silicon Valley ethos of moving fast and breaking things, many organizations have been rolling out technology without much concern for existing rules and regulations.

Uber, Airbnb, and the myriad e-scooter startups are on the back foot all over Europe as the state reasserts its authority. Even in the U.S., regulators have started to put their foot down. Tesla is having to reprogram 50,000 vehicles that were intentionally programmed to disrespect stop signs. If the car was driving slowly and couldn’t see anybody else around an intersection, it would ignore the stop sign and continue into the intersection. That’s illegal, but humans do it all the time. It turns out authorities were less than thrilled to see bad human behavior programmed into Tesla’s cars.

We have rules for a reason. Some of them are ridiculous (like the ubiquitous cooking consent), but good citizenship includes adhering to the rules until you can persuade the rule-maker to change them. Don’t be like Tesla.

Bad Face Recognition

Spiritual Programmer, ,

In the U.S., police have started rounding up suspects based on defective image recognition from grainy surveillance video. Image recognition is known to work poorly on black faces, probably because they were mainly trained on images of white people.

When we roll out new technology, we of carefully explain to the users where and how it can be used. But if we can reasonably expect our users to ignore our admonitions, maybe we shouldn’t sell it at all.

https://www.theverge.com/2021/4/13/22382398/robert-williams-detroit-police-department-aclu-lawsuit-facial-recognition-wrongful-arrest

Doing the Right Thing

Beneficial Intelligence,

Last week’s episode of my podcast Beneficial Intelligence was about doing the right thing. Google used to say “Don’t be evil,” but now they are struggling with their employees who want them to do the right thing. Amazon is unpopular for squeezing warehouse workers, and McKinsey paid $600 million for the role their advice played in the opioid epidemic in the U.S. They could have done the right thing, but didn’t.

As CIO, you also constantly have opportunities to cut corners and squeeze employees to work a little harder. But if you want to attract and retain top talent, you need to do the right thing. 

Doing the Right Thing

Beneficial Intelligence,

This week’s episode of my podcast Beneficial Intelligence is about doing the right thing. Google started out with a motto of “Don’t be evil” but that has fallen by the wayside. Occasionally, employees can enforce a change as when they stopped working on military AI. But Google doesn’t seem terribly committed, and their Ethical AI Team is falling apart after they fired the head researcher.

Amazon never promised not to be evil, and they are forcing their delivery drivers to do 10-hour graveyard shifts starting before sunrise and going until mid-day. They are trying to avoid tired drivers causing accidents by installing cameras and AI in the vans so the computer can detect when the worker is falling asleep behind the wheel and can wake him up.

As a CIO, you’re engaged in a war for talent. But you also need to meet your budget, implement hot new technologies like AI and maintain IT security. There is always an opportunity to cut a corner, roll out inadequately tested technology or squeeze employees so you can hit your goals this quarter. But if you want to be able to attract and keep top IT talent, you need to do the right thing.

Listen here or find “Beneficial Intelligence” wherever you get your podcasts.