Why a Computer Scientist Thinks Superintelligence Won’t Happen

Advertisement
Excerpt Spectrum IEEE spectrum.ieee.org

Many people were caught off guard this past summer when employees of one of the world’s leading AI companies began casually debating whether their products would lead to human extinction. It started when Anthropic researcher Jacob Coxon announced his resignation, warning that his employer and OpenAI were “racing straight to self-improving superintelligence and gambling with our lives.” Evan Hubinger, Anthropic’s head of alignment science, replied, “Jacob is correct here—we really do earnestly believe AI could kill all humans!” And Anthropic CEO Dario Amodei subsequently told CNN, “I agree with Jacob much more than I disagree with him.”

These concerns stem from a belief that has circulated in futurist circles for many decades: Once engineers design an AI system capable of improving itself, it will trigger an accelerating cycle of autonomous upgrades known as recursive self-improvement (RSI). The result will be artificial superintelligence (ASI), a system with cognitive capabilities so vastly exceeding those of humans that it will easily evade our control and perhaps even wipe out our species. Artificial superintelligence is, it should be noted, a far more ambitious goal than artificial general intelligence (AGI), which generally refers to AI that can match human capabilities across a broad range of cognitive tasks.

These ideas have prompted varied reactions. Some argue that the concept of ASI is being used for marketing or to instigate regulation that will help the incumbent labs squash new competitors. Others focus on the plausibility of such systems killing us all, debating the finer points of specific scenarios such as AI-engineered superviruses. Still others claim that aligning ASI to human interests is feasible and so we should rush ahead to usher in this digital utopia.

Amid these conversations, one question has received comparatively little attention: From an engineering perspective, is superintelligence even possible? The answer is important because it can help us calibrate our anxieties and develop the most effective possible responses to this emerging technology.

Continue Reading at Spectrum IEEE

Comments

VIP

Join the Conversation

VIP members get the ability to comment on articles.

Recommended

Trending on HotAir Videos