| Both sides previous revision
Previous revision
Next revision
|
Previous revision
|
uncategorized:ai_risk_surveys [2023/05/10 02:03]
zachsteinperlman |
uncategorized:ai_risk_surveys [2024/08/12 19:34] (current)
katjagrace |
| ====== AI Risk Surveys ====== | /* |
| | EDITOR COMMENTS: |
| |
| //Published 9 May 2023// | -Harlan: we need to add details about the 2023 survey here, and the generation lab thing |
| | */ |
| | ====== Surveys of experts on levels of AI Risk ====== |
| |
| We know of four surveys of AI experts and two surveys of AI safety/governance experts on risks from advanced AI. | //Published 9 May 2023; last updated 23 May 2023// |
| |
| ===== Details ===== | //This page is being updated, and may be low quality.// |
| |
| ==== Surveys of AI experts ==== | We know of six surveys of AI experts and two surveys of AI safety/governance experts on risks from advanced AI. |
| | |
| | ===== Surveys of AI experts ===== |
| | |
| | ==== 2016 Expert Survey on Progress in AI ==== |
| | |
| | //(Main article: [[ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:2016_expert_survey_on_progress_in_ai|2016 Expert Survey on Progress in AI]])// |
| | |
| | Paper: [[https://jair.org/index.php/jair/article/view/11222|When Will AI Exceed Human Performance? Evidence from AI Experts]] (Grace et al. 2016, published 2018) |
| |
| * [[https://jair.org/index.php/jair/article/view/11222|When Will AI Exceed Human Performance? Evidence from AI Experts]] (Grace et al. 2016, published 2018) | |
| * "Say we have '**high-level machine intelligence**' when unaided machines can accomplish every task better and more cheaply than human workers. Ignore aspects of tasks for which being a human is intrinsically advantageous, e.g. being accepted as a jury member. //Think feasibility, not adoption.// Assume for the purpose of this question that HLMI will at some point exist. How positive or negative do you expect the overall impact of this to be on humanity, in the long run? Please answer by saying how probable you find the following kinds of impact, with probabilities adding to 100%:" | * "Say we have '**high-level machine intelligence**' when unaided machines can accomplish every task better and more cheaply than human workers. Ignore aspects of tasks for which being a human is intrinsically advantageous, e.g. being accepted as a jury member. //Think feasibility, not adoption.// Assume for the purpose of this question that HLMI will at some point exist. How positive or negative do you expect the overall impact of this to be on humanity, in the long run? Please answer by saying how probable you find the following kinds of impact, with probabilities adding to 100%:" |
| * "Extremely good (e.g. rapid growth in human flourishing)": median 20% | * "Extremely good (e.g. rapid growth in human flourishing)": median 20% |
| * "Extremely bad (e.g. human extinction)": median 5% | * "Extremely bad (e.g. human extinction)": median 5% |
| * 40% of responses had at least 10% on "extremely bad" | * 40% of responses had at least 10% on "extremely bad" |
| | |
| | |
| * "Stuart Russell's argument" | * "Stuart Russell's argument" |
| * Respondents were presented with an excerpt from a piece by Stuart Russell, then asked "Do you think this argument points at an important problem?" | * Respondents were presented with an excerpt from a piece by Stuart Russell, then asked "Do you think this argument points at an important problem?" |
| * 34%: "Yes, an important problem" | * 34%: "Yes, an important problem" |
| * 5%: "Yes, among the most important problems in the field" | * 5%: "Yes, among the most important problems in the field" |
| | |
| | |
| * AI safety research | * AI safety research |
| * Respondents were presented with a definition of "AI safety research," then asked "How much should society prioritize **AI safety research**, relative to how much it is currently prioritized?" | * Respondents were presented with a definition of "AI safety research," then asked "How much should society prioritize **AI safety research**, relative to how much it is currently prioritized?" |
| * 5% "much less"; 8% "less"; 38% "about the same as it is now"; 35% "more"; 14% "much more"((Source: [[https://aiimpacts.org/2016-expert-survey-on-progress-in-ai/|AI Impacts]]. This contradicts [[https://jair.org/index.php/jair/article/view/11222/26431|JAIR]], which says 5% "much less"; 6% "less"; 41% "about the same as it is now"; 35% "more"; 12% "much more".)) | * 5% "much less"; 8% "less"; 38% "about the same as it is now"; 35% "more"; 14% "much more"((Source: [[https://aiimpacts.org/2016-expert-survey-on-progress-in-ai/|AI Impacts]]. This contradicts [[https://jair.org/index.php/jair/article/view/11222/26431|JAIR]], which says 5% "much less"; 6% "less"; 41% "about the same as it is now"; 35% "more"; 12% "much more".)) |
| | |
| | |
| * Population: authors of papers at ICML or NeurIPS 2015 | * Population: authors of papers at ICML or NeurIPS 2015 |
| * The survey was sent to 1634 people and received 352 responses. | * The survey was sent to 1634 people and received 352 responses. |
| | |
| | ==== Zhang et al 2019 ==== |
| | |
| * [[https://arxiv.org/pdf/2206.04132.pdf#page=24|Forecasting AI Progress: Evidence from a Survey of Machine Learning Researchers]] (Zhang et al. 2019, published 2022) | * [[https://arxiv.org/pdf/2206.04132.pdf#page=24|Forecasting AI Progress: Evidence from a Survey of Machine Learning Researchers]] (Zhang et al. 2019, published 2022) |
| * Long-run impact of high-level machine intelligence | * Long-run impact of high-level machine intelligence |
| * Population: authors of papers at ICML or NeurIPS 2018 | * Population: authors of papers at ICML or NeurIPS 2018 |
| * The survey was sent to 2652 people and received 524 responses. | * The survey was sent to 2652 people and received 524 responses. |
| * [[https://wiki.aiimpacts.org/doku.php?id=ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:2022_expert_survey_on_progress_in_ai|2022 Expert Survey on Progress in AI]] (Grace et al. 2022, publication forthcoming in 2023) | |
| | ==== 2022 Expert Survey on Progress in AI ==== |
| | |
| | |
| | //(Main article: [[ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:2022_expert_survey_on_progress_in_ai|2022 Expert Survey on Progress in AI]])// |
| | |
| | * [[https://wiki.aiimpacts.org/doku.php?id=ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:2022_expert_survey_on_progress_in_ai|2022 Expert Survey on Progress in AI]] (Grace et al. 2022) |
| | |
| * Extinction | * Extinction |
| * "What probability do you put on future AI advances causing human extinction or similarly permanent and severe disempowerment of the human species? | * "What probability do you put on future AI advances causing human extinction or similarly permanent and severe disempowerment of the human species? |
| * Population: authors of papers at ICML or NeurIPS 2021 | * Population: authors of papers at ICML or NeurIPS 2021 |
| * The survey was sent to "approximately 4271" people and received 738 responses. | * The survey was sent to "approximately 4271" people and received 738 responses. |
| | |
| | ==== Michael et al 2022 ==== |
| | |
| * [[https://arxiv.org/pdf/2208.12852.pdf#page=10|What Do NLP Researchers Believe? Results of the NLP Community Metasurvey]] (Michael et al. 2022) | * [[https://arxiv.org/pdf/2208.12852.pdf#page=10|What Do NLP Researchers Believe? Results of the NLP Community Metasurvey]] (Michael et al. 2022) |
| * Nuclear-level catastrophe | * Nuclear-level catastrophe |
| * Population: "researchers who publish at computational linguistics conferences." See pp. 3–4 for details. | * Population: "researchers who publish at computational linguistics conferences." See pp. 3–4 for details. |
| * "We compute that 6323 people [published at least two papers at computational linguistics conferences] during the survey period according to publication data in the ACL Anthology, meaning we have survey responses from about 5% of the total." | * "We compute that 6323 people [published at least two papers at computational linguistics conferences] during the survey period according to publication data in the ACL Anthology, meaning we have survey responses from about 5% of the total." |
| | |
| | ==== Generation Lab 2023 ==== |
| | |
| | * [[https://www.generationlab.org/axios-generationlab-syracuse| |
| | AI EXPERT SURVEY (n=216 computer science professors)]] (Generation Lab, 2023) |
| | |
| | ==== Expert Survey on Progress in AI 2023 ==== |
| | |
| | * [[https://wiki.aiimpacts.org/ai_timelines/predictions_of_human-level_ai_timelines/ai_timeline_surveys/2023_expert_survey_on_progress_in_ai|2023 Expert Survey on Progress in AI]] (Grace et al. 2023) |
| |
| === Not currently included on this list === | === Not currently included on this list === |
| * The informal [[https://wiki.aiimpacts.org/doku.php?id=ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:kruel_ai_interviews|Alexander Kruel interviews]] from 2011–2012. | * The informal [[https://wiki.aiimpacts.org/doku.php?id=ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:kruel_ai_interviews|Alexander Kruel interviews]] from 2011–2012. |
| * Ezra Karger and Philip Tetlock et al.'s "Hybrid Forecasting-Persuasion Tournament" (2022, results to be released around 1 June 2023). "The median AI expert gave a 3.9% chance to an existential catastrophe (where fewer than 5,000 humans survive) owing to AI by 2100" ([[https://www.economist.com/science-and-technology/2023/04/19/how-generative-models-could-go-wrong|The Economist]]). We will know more when the report is out. We are tentatively concerned about population quality and sampling bias. In particular, Zach Stein-Perlman was invited to participate as an AI expert in May 2022; he was not an AI expert. | * Ezra Karger and Philip Tetlock et al.'s "Hybrid Forecasting-Persuasion Tournament" (2022, results to be released around 1 June 2023). "The median AI expert gave a 3.9% chance to an existential catastrophe (where fewer than 5,000 humans survive) owing to AI by 2100" ([[https://www.economist.com/science-and-technology/2023/04/19/how-generative-models-could-go-wrong|The Economist]]). We will know more when the report is out. We are tentatively concerned about population quality and sampling bias. In particular, Zach Stein-Perlman was invited to participate as an AI expert in May 2022; he was not an AI expert. |
| |
| |
| ==== Surveys of AI safety/governance experts ==== | ==== Surveys of AI safety/governance experts ==== |
| * Population: "prominent AI safety and governance researchers" | * Population: "prominent AI safety and governance researchers" |
| * "We sent the survey to 135 researchers at leading AI safety/governance research organisations (including AI Impacts, CHAI, CLR, CSER, CSET, FHI, FLI, GCRI, MILA, MIRI, Open Philanthropy and PAI) and a number of independent researchers. We received 75 responses, a response rate of 56%." | * "We sent the survey to 135 researchers at leading AI safety/governance research organisations (including AI Impacts, CHAI, CLR, CSER, CSET, FHI, FLI, GCRI, MILA, MIRI, Open Philanthropy and PAI) and a number of independent researchers. We received 75 responses, a response rate of 56%." |
| | |
| | === Not currently included on this list === |
| | * [[https://www.conjecture.dev/timelines-and-pdoom/|Conjecture internal survey: AGI timelines and probability of human extinction from advanced AI]] (Sala 2023). Note that being more worried about AI risk makes someone more likely to work at Conjecture. |
| |
| ==== Other ==== | ==== Other ==== |
