It will not be technology or regulations that limit AI; it will be insurance. Several major U.S. insurance companies are petitioning lawmakers to allow them to completely exclude AI risks from their coverage, arguing that they pose an unmanageable risk.
Insurance companies are already fighting with Air Canada over who will pay for the fictitious discounts that their chatbot invented and they had to honor. Next time something like that happens, AI will allow thousands of customers to easily create plausible claims, but the insurance company cannot use AI to handle them. This asymmetry has them scared, for good reason.
If you are running AI systems, take a good look at your Tech E&O insurance. You are likely to find that it already limits coverage from some types of AI incidents. When it comes up for renewal, you will find more AI excluded. Insurance is just as real a limitation as regulation and technology.
