SA: I mean, we can take any sort of trope that we want here. What if we’re able to cure every disease? That would be a huge victory on its own. What if every person on Earth can have a better education than any person on Earth gets today? That would be pretty good. What if every person a hundred years from now is a hundred times richer in the subjective sense? Maybe they’re happier, healthier, have more material possessions, more ability to live the good life in the way it’s assigned to them than people are today. I think all of these things are realistically possible.
BW: So, what’s the other side of it? You said the worst-case scenario is “lights out for all of us.” I’m sure a lot of people have quoted that line back to you. What did you mean by it?
SA: I understand why people would be more comfortable if I would only talk about the great future here, and I do think that’s what we’re going to get. I think this can be managed. I also think the more that we talk about the potential downsides, the more that we as a society work together on how we want this to go, it’s much more likely that we’re going to be in the upside case. But if we pretend like there is not a pretty serious misuse case here and just say, “Full steam ahead! It’s all great! Don’t worry about anything!”—I just don’t think that’s the right way to get to the good outcome. When we were developing nuclear technology, we didn’t just say, “Hey, this is so great, we can power the world! Oh yeah, don’t worry about that bomb thing. It’s never going to happen.” Instead, the world really grappled with that, and I think we’ve gotten to a surprisingly good place.
[I’m old enough to recognize that any new, emerging technology with significant disruptive force starts off looking like the end of the world, and then never is. If you want to see how this works, go back and look at 1980s films like “Johnny Mnemonic,” which not only captured the panic over networked computers but also the panic over the Japanization of America. (And it was hardly alone, but it was still a fun if low-brow film.) One of these days, though, the prediction may end up being right. — Ed]
Join the conversation as a VIP Member