On October 31, the Defense Innovation Board unveiled principles for the ethical use of AI by the Defense Department, which call for AI systems in the military to be responsible, equitable, reliable, traceable, and governable. Though the recommendations are non-binding, the Department is likely to implement a substantial number of them. The special focus on AI-based weapons arises because their speed and precision make them indispensable in modern warfare. Meanwhile, their novel elements create new and substantial risks that must be managed successfully to take advantage of these new capabilities.
WHAT MAKES AI WEAPONS SYSTEMS SO CONTROVERSIAL?
The chief concern of the Board was the possibility that an AI weapon system might not perform as intended, with potentially catastrophic results. Machine learning incorporated into weapons systems might learn to carry out unintended attacks on targets that the military had not approved and escalate a conflict. They might in some other way escape from the area of use for which they had been designed and launch with disastrous outcomes.