There are two broad classes of methods for how one might keep an artificial super-intelligence from destroying the world, Bostrom says. One involves controlling an AI’s capabilities–perhaps by preventing it from having access to the Internet or by not giving it any physical manipulators such as robotic arms.

While limiting what an artificial super-intelligence might do could be useful in the initial stages of developing such a machine, “we can’t expect to keep a super-intelligent genie locked up in its bottle forever, or even for a short time,” Bostrom says. For instance, an artificial super-intelligence might develop ways to trick any human gatekeepers to let it out of its “box.” Human beings are not secure systems, especially not when pitched against a super-intelligent schemer, he notes.

Instead, Bostrom advises shaping what artificial super-intelligences want to do so that even if they were able to cause great harm, they would choose not to do so. One strategy would involve directly specifying a set of rules for an AI to follow, such as Isaac Asimov’s famed Three Laws of Robotics. However, this poses the challenge of choosing which rules we would want to guide the AI and the difficulty of expressing those values in computer code.

A second alternative involves giving an AI only modest goals and limited ambitions. However, care would have to be taken in defining how the AI should minimize its impact on the world.