ع
The Expert Survey on Progress in AI conducted by AI Impacts in December 2024 and released this month reached respondents who had published at six leading AI venues with a 10 percent response rate. AI Impacts data places the average probability assigned to future AI advances causing human extinction or similarly permanent and severe disempowerment at 18 percent with the median rising to 10 percent from 5 percent in earlier iterations of the survey. The organisation’s assessment found that 81 percent of participants gave such outcomes at least a 1 percent chance while 12 percent assigned zero probability and roughly one third estimated 20 percent or higher. Seventy two percent of those surveyed favored greater prioritisation of research aimed at minimising potential AI risks.
Figures from the same AI Impacts survey show that expectations for when high level machine intelligence might emerge have moved earlier since previous rounds with substantial uncertainty remaining about long term consequences. A paper drawing on the data and published on arXiv noted that between 38 and 51 percent of respondents gave advanced AI at least a 10 percent chance of producing outcomes as bad as human extinction. The analysis indicated that while 68 percent of participants viewed good outcomes from superhuman AI as more likely than bad a significant share of those optimists still assigned at least 5 percent probability to extremely bad results.
According to a New York Times report many researchers believe narratives focused on machine god style doomsday scenarios have come to dominate discourse despite representing a relatively small group within the field. The newspaper stated that critics such as Timnit Gebru argue the emphasis on existential risks serves to accelerate development while diverting attention from harms already occurring including AI guided weapons data centre contributions to climate change and automation used to displace workers. Business Insider quoted computer scientist Gary Marcus as saying there are some very serious harms to worry about but no realistic scenario that he was aware of for actually killing all humans.
An aggregation of publicly stated estimates from more than 50 AI researchers forecasters and organisations between 2020 and 2026 compiled by Calcuja placed the median probability of existential catastrophe from AI at around 20 percent. The Calcuja review showed wide variation with mainstream machine learning researchers clustering near 5 percent safety focused researchers near 30 percent and individual estimates ranging from near zero by figures such as Yann LeCun to above 90 percent by some aligned with effective altruism. Calcuja data indicated that tech industry leaders tended to fall around 15 percent in their median estimates.
The BBC reported that computational linguist Sasha Luccioni views certain AI risks as important but concentrates on measurable problems such as the carbon emissions from training models and the potential for systems to amplify societal biases. In remarks carried by the broadcaster University of Alberta professor Mark Riedl contended that current AI lacks the agency or intrinsic goals that would be necessary for it to pursue scenarios involving takeover or extinction. The BBC article highlighted that such perspectives coexist with calls from other prominent voices including Geoffrey Hinton and Yoshua Bengio for treating extinction level risks with the same urgency as pandemics or nuclear war.
AI Impacts has run versions of the survey since 2016 and its latest edition noted that the profile of perceived risks has remained relatively stable even as the technology has moved from niche research to powering the world’s most valuable companies. The organisation’s report recorded that most participants continue to express substantial uncertainty about AI timelines and impacts with more than half voicing at least moderate concern over nearer term issues such as misinformation authoritarian control and economic inequality. A separate arXiv analysis of the responses underscored that the median view across thousands of researchers supports at least a 5 percent chance of extinction level events and described the finding as robust across different question phrasings.
