Sten Vesterli's Blog

Making IT Live up to its Promise

Tag: Failure

Another Place Not to Use AI Chatbots

IT Leadership, , ,

Alaska wanted an AI chatbot to give legal advice. Anybody care to guess how that went?

Yes, not well. They are now 15 months into a 3-month project, but expect it to go live this month.

I’ll make a prediction: This project will end up in the bucket of IT projects that sink without a trace, leaving only cost and no business benefit whatsoever. Sadly, one in four IT projects still end in that bucket. My hunch is that this number is increasing, as AI is incorrectly applied to more and more use cases.

If you are going to implement AI with an LLM, don’t do it in a critical application like giving legal advice to citizens. There is no way to stop an LLM from hallucinating. The only way to automate advice and be sure it is correct is with old-school expert systems.

Are You Afraid Robots Will Take Your Job?

IT Leadership, Technology That Fits, , ,

Robots are not taking our jobs. It’s a good story to create eye-catching headlines and generate clicks, but the numbers do not support it in any way.  Michael Handel of the U.S. Bureau of Labor Statistics has published a paper where he carefully analyzes job losses across many professions. He finds that job losses follow long-term trends, and there is no hint of the dramatic changes predicted by people who make a living from predicting that the sky will shortly fall.

That matches what I see in the organizations I work with. Traditional IT projects regularly fail, and AI projects have an even higher failure rate. They might deliver something, but too often, it turns out to be impossible to move an AI experiment out of the lab and into productive use.

Additionally, in the cases where AI does provide real business benefits, it handles one specific task and not a whole job. All of our AI today is very narrowly trained for one task. That frees up workers to do more useful things with their time, making them more productive.

For example, the illustration for this post is made by me and the Midjourney AI. It was told to illustrate “the robots are not taking our jobs.” We ran a few iterations where I selected the best of its suggestions until we came up with this image.