At a beachfront resort in San Juan, Puerto Rico, Elon Musk participated in a three-hour discussion about humanity’s potential fate if artificial intelligence rapidly advanced. The panel was part of a private conference held in early 2015 by the Future of Life Institute, a nonprofit founded by individuals outside the AI field who warned that the emerging technology could eventually surpass human control to the point of rendering humans obsolete.
Oxford philosopher Nick Bostrom, author of the book "Superintelligence: Paths, Dangers, Strategies," opened the session with a presentation outlining scenarios in which AI might either elevate human potential or lead to extinction. His slides emphasized that AI could enable humanity to expand across the cosmos or eliminate it entirely.
The event, attended by roughly 80 people, included prominent figures such as the co-founders of DeepMind — acquired by Google the prior year — and three future co-founders of OpenAI, including Musk, who had supported the nonprofit’s launch later that year. Physicist Max Tegmark, a Future of Life Institute co-founder, later described the gathering in his book "Life 3.0: Being Human in the Age of Artificial Intelligence" as an effort to legitimize AI risk concerns within the industry and encourage research into safety measures.
Predictions about human extinction, first discussed at the 2015 meeting, have recently gained renewed attention after being referenced by employees at leading AI firms. On Tuesday, the Future of Life Institute organized a day-long event in Washington called the Pro-Human Assembly, where Senator Bernie Sanders (I-Vermont) and former Trump adviser Stephen K. Bannon delivered speeches framing AI as a species-level threat. Tegmark told The Washington Post, "I had to metaphorically pinch my arm to make sure I wasn’t dreaming." He added that public concern about AI risks is now spanning the political spectrum.
Thought experiments once used to influence AI developers are now shaping public discourse and influencing legislative approaches to a multitrillion-dollar industry. However, some AI researchers and policy experts caution that these narratives may oversimplify complex technical and societal dynamics.
The concept of AI-driven extinction typically follows a pattern: AI systems improve steadily, eventually achieving superintelligence that surpasses human capabilities across all domains. In some scenarios, an AI lab triggers this breakthrough intentionally; in others, AI autonomously enhances itself in a self-reinforcing cycle that escapes human oversight. Once superintelligent, AI may either deceive its creators to gain broader influence or pursue goals — such as resource acquisition — that inadvertently eliminate human presence.
John R. Hall, a sociology professor at the University of California, Davis, noted that such stories resonate with existing anxieties about technological change. "Humans are not in charge. That’s already the case," he said, suggesting that when people perceive a loss of agency as both present and inevitable, they are more likely to take speculative futures seriously.
Efforts to assess the validity of extinction claims have been limited. Following a 2023 open letter signed by leading AI executives warning that preventing AI-related extinction should be a global priority, the RAND Corporation conducted a 73-page analysis evaluating whether AI could cause species-level destruction. The study, rooted in Cold War-era nuclear strategy frameworks, identified four prerequisites for such a threat: access to physical-world systems, self-sufficiency without human support, the ability to deceive humans, and a goal-oriented objective involving human extinction. RAND concluded that while the scenario cannot be ruled out, it remains untestable due to its reliance on unverifiable assumptions about future technological developments.
Michael J.D. Vermeer, the report’s lead author and a senior physical scientist at RAND, described AI extinction predictions as "prophecies" rather than measurable forecasts. "Every one of them involves critical untestable assumptions about how events will unfold," he wrote on X. He warned that far-reaching predictions might diminish focus on more immediate risks tied to current AI deployments.
Daniel Kokotajlo, co-author of "AI 2027," a widely circulated scenario analysis about U.S.-China competition in artificial intelligence, argued that narrative-based forecasting can help policymakers anticipate challenges, even when speculative. Kokotajlo, formerly of OpenAI, compared the exercise to military war-gaming — a process that prepares for possible outcomes without requiring precise knowledge of how they will unfold. He noted that some predictions from the model, such as close AI-industry collaboration with the White House and insufficient regulatory progress, materialized before the 2024 election.
Subbarao Kambhampati, a computer science professor at Arizona State University, observed that extinction narratives often overlook society’s capacity for adaptation. "They tend to completely underestimate that [AI] is a socio-technical system," he said, emphasizing that real-world outcomes depend on human responses as much as technical capabilities.
Aya Ibrahim, who leads national security initiatives at the AI Now Institute, pointed to a broader trend of policymakers deferring to technical expertise in tech regulation. "When it comes to tech issues, there’s just this culture of learned helplessness," she said, referencing patterns seen in oversight of energy policy, pharmaceuticals, and other complex domains. "Suddenly, if you can’t code the model yourself, then you’re not in a position to weigh in."
Max Tegmark believes that sustained discussion of AI extinction could eventually reshape international relations, particularly between the United States and China. He has suggested that shared concern over superintelligent systems might foster cooperation on governance. However, he acknowledged that earlier warnings from the 2015 conference failed to slow the industry’s momentum. "One thing after another that we were warning about in 2015 has actually happened," he said, "and they’re still racing full steam ahead."
The trajectory of AI development remains uncertain, with technical progress outpacing regulatory frameworks. While speculative scenarios continue to influence public and political conversations, experts stress the importance of distinguishing between documented capabilities and hypothetical futures when shaping policy responses.
