What differentiates a professional software organization from a bunch of amateurs? One thing is the ability to roll back.
It’s not a good day at the office when the Federal Aviation Authority issues an Emergency Airworthiness Directive, grounding 6,000 of the aircraft your customers are flying. A JetBlue flight on autopilot suddenly turned nose-down in mid-flight, and it turns out that the L104 version of the ELAC software was vulnerable to memory corruption due to solar radiation.
But aircraft manufacturers and airlines have procedures in place, and engineers rapidly fanned out to airports around the world, rolling back to the L103 version.
It is impossible to test every situation, and every once in a while, something unforeseen happens. Professional software organizations can rapidly recover. Do you have rollback plans in place every time you roll out new versions of critical software?