Safety critical systems (SCS) are the only real benchmark for ML systems. Thoughts? [D]

What are real-world safety critical systems (SCS)?

• A flight controller for a commercial airplane carrying 300 passengers.
• A braking system for a bullet train that operates at 320km/hour.
• A reactor protection system for nuclear power plant that serves millions of people.
• A piece of medical equipment that regulate certain bodily rhythm for a patient.
• A railway crossing system for a network involving dozens of trains in a large city.
• ...

I believe that if ML systems, built off of LLM and NN based methods, can work in these safety critical systems, then it can sway a lot of people who don't believe in the technology while solving multiple problems facing ML field at the moment, such as:

• Too many papers being produced that works well on test sets and various benchmarks, but says nothing about real-world performance. If it doesn't work for SCS, then it doesn't work. This cuts down the amount of nonreproducible papers and overclaiming.
• Too many simulations that don't work outside of the simulator. Again, same as the above.
• Too many AI companies claiming that their model is the work of God. Ok, then put the model to the test by making it run the ramping and discharging process of a nuclear reactor that serves millions of people. Just let the nuclear reactor do what the LLM tells it to do!
• People within ML and in other traditional areas of engineering think AI/ML is all hype, alchemy and snake-oil. There is nothing better to convince the nonbeliever than a Boeing-737 airplane with 230 passenger that flies purely off of LLM as controller + ConvNet as sensor or using some VLM/VLA/VLN technology.

Is this proposal too radical for ML in 2026?