| Both sides previous revision
Previous revision
Next revision
|
Previous revision
|
uncategorized:ai_risk_surveys [2024/01/09 20:04]
harlanstewart |
uncategorized:ai_risk_surveys [2024/08/12 19:34] (current)
katjagrace |
| EDITOR COMMENTS: | EDITOR COMMENTS: |
| |
| -Harlan: we need to add details about the 2023 survey here | -Harlan: we need to add details about the 2023 survey here, and the generation lab thing |
| */ | */ |
| ====== AI Risk Surveys ====== | ====== Surveys of experts on levels of AI Risk ====== |
| |
| //Published 9 May 2023; last updated 23 May 2023// | //Published 9 May 2023; last updated 23 May 2023// |
| |
| We know of five surveys of AI experts and two surveys of AI safety/governance experts on risks from advanced AI. | //This page is being updated, and may be low quality.// |
| |
| ===== Details ===== | We know of six surveys of AI experts and two surveys of AI safety/governance experts on risks from advanced AI. |
| |
| ==== Surveys of AI experts ==== | ===== Surveys of AI experts ===== |
| | |
| | ==== 2016 Expert Survey on Progress in AI ==== |
| | |
| | //(Main article: [[ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:2016_expert_survey_on_progress_in_ai|2016 Expert Survey on Progress in AI]])// |
| | |
| | Paper: [[https://jair.org/index.php/jair/article/view/11222|When Will AI Exceed Human Performance? Evidence from AI Experts]] (Grace et al. 2016, published 2018) |
| |
| * [[https://jair.org/index.php/jair/article/view/11222|When Will AI Exceed Human Performance? Evidence from AI Experts]] (Grace et al. 2016, published 2018) | |
| * "Say we have '**high-level machine intelligence**' when unaided machines can accomplish every task better and more cheaply than human workers. Ignore aspects of tasks for which being a human is intrinsically advantageous, e.g. being accepted as a jury member. //Think feasibility, not adoption.// Assume for the purpose of this question that HLMI will at some point exist. How positive or negative do you expect the overall impact of this to be on humanity, in the long run? Please answer by saying how probable you find the following kinds of impact, with probabilities adding to 100%:" | * "Say we have '**high-level machine intelligence**' when unaided machines can accomplish every task better and more cheaply than human workers. Ignore aspects of tasks for which being a human is intrinsically advantageous, e.g. being accepted as a jury member. //Think feasibility, not adoption.// Assume for the purpose of this question that HLMI will at some point exist. How positive or negative do you expect the overall impact of this to be on humanity, in the long run? Please answer by saying how probable you find the following kinds of impact, with probabilities adding to 100%:" |
| * "Extremely good (e.g. rapid growth in human flourishing)": median 20% | * "Extremely good (e.g. rapid growth in human flourishing)": median 20% |
| * "Extremely bad (e.g. human extinction)": median 5% | * "Extremely bad (e.g. human extinction)": median 5% |
| * 40% of responses had at least 10% on "extremely bad" | * 40% of responses had at least 10% on "extremely bad" |
| | |
| | |
| * "Stuart Russell's argument" | * "Stuart Russell's argument" |
| * Respondents were presented with an excerpt from a piece by Stuart Russell, then asked "Do you think this argument points at an important problem?" | * Respondents were presented with an excerpt from a piece by Stuart Russell, then asked "Do you think this argument points at an important problem?" |
| * 34%: "Yes, an important problem" | * 34%: "Yes, an important problem" |
| * 5%: "Yes, among the most important problems in the field" | * 5%: "Yes, among the most important problems in the field" |
| | |
| | |
| * AI safety research | * AI safety research |
| * Respondents were presented with a definition of "AI safety research," then asked "How much should society prioritize **AI safety research**, relative to how much it is currently prioritized?" | * Respondents were presented with a definition of "AI safety research," then asked "How much should society prioritize **AI safety research**, relative to how much it is currently prioritized?" |
| * 5% "much less"; 8% "less"; 38% "about the same as it is now"; 35% "more"; 14% "much more"((Source: [[https://aiimpacts.org/2016-expert-survey-on-progress-in-ai/|AI Impacts]]. This contradicts [[https://jair.org/index.php/jair/article/view/11222/26431|JAIR]], which says 5% "much less"; 6% "less"; 41% "about the same as it is now"; 35% "more"; 12% "much more".)) | * 5% "much less"; 8% "less"; 38% "about the same as it is now"; 35% "more"; 14% "much more"((Source: [[https://aiimpacts.org/2016-expert-survey-on-progress-in-ai/|AI Impacts]]. This contradicts [[https://jair.org/index.php/jair/article/view/11222/26431|JAIR]], which says 5% "much less"; 6% "less"; 41% "about the same as it is now"; 35% "more"; 12% "much more".)) |
| | |
| | |
| * Population: authors of papers at ICML or NeurIPS 2015 | * Population: authors of papers at ICML or NeurIPS 2015 |
| * The survey was sent to 1634 people and received 352 responses. | * The survey was sent to 1634 people and received 352 responses. |
| | |
| | ==== Zhang et al 2019 ==== |
| | |
| * [[https://arxiv.org/pdf/2206.04132.pdf#page=24|Forecasting AI Progress: Evidence from a Survey of Machine Learning Researchers]] (Zhang et al. 2019, published 2022) | * [[https://arxiv.org/pdf/2206.04132.pdf#page=24|Forecasting AI Progress: Evidence from a Survey of Machine Learning Researchers]] (Zhang et al. 2019, published 2022) |
| * Long-run impact of high-level machine intelligence | * Long-run impact of high-level machine intelligence |
| * Population: authors of papers at ICML or NeurIPS 2018 | * Population: authors of papers at ICML or NeurIPS 2018 |
| * The survey was sent to 2652 people and received 524 responses. | * The survey was sent to 2652 people and received 524 responses. |
| * [[https://wiki.aiimpacts.org/doku.php?id=ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:2022_expert_survey_on_progress_in_ai|2022 Expert Survey on Progress in AI]] (Grace et al. 2022, publication forthcoming in 2023) | |
| | ==== 2022 Expert Survey on Progress in AI ==== |
| | |
| | |
| | //(Main article: [[ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:2022_expert_survey_on_progress_in_ai|2022 Expert Survey on Progress in AI]])// |
| | |
| | * [[https://wiki.aiimpacts.org/doku.php?id=ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:2022_expert_survey_on_progress_in_ai|2022 Expert Survey on Progress in AI]] (Grace et al. 2022) |
| | |
| * Extinction | * Extinction |
| * "What probability do you put on future AI advances causing human extinction or similarly permanent and severe disempowerment of the human species? | * "What probability do you put on future AI advances causing human extinction or similarly permanent and severe disempowerment of the human species? |
| * Population: authors of papers at ICML or NeurIPS 2021 | * Population: authors of papers at ICML or NeurIPS 2021 |
| * The survey was sent to "approximately 4271" people and received 738 responses. | * The survey was sent to "approximately 4271" people and received 738 responses. |
| | |
| | ==== Michael et al 2022 ==== |
| | |
| * [[https://arxiv.org/pdf/2208.12852.pdf#page=10|What Do NLP Researchers Believe? Results of the NLP Community Metasurvey]] (Michael et al. 2022) | * [[https://arxiv.org/pdf/2208.12852.pdf#page=10|What Do NLP Researchers Believe? Results of the NLP Community Metasurvey]] (Michael et al. 2022) |
| * Nuclear-level catastrophe | * Nuclear-level catastrophe |
| * Population: "researchers who publish at computational linguistics conferences." See pp. 3–4 for details. | * Population: "researchers who publish at computational linguistics conferences." See pp. 3–4 for details. |
| * "We compute that 6323 people [published at least two papers at computational linguistics conferences] during the survey period according to publication data in the ACL Anthology, meaning we have survey responses from about 5% of the total." | * "We compute that 6323 people [published at least two papers at computational linguistics conferences] during the survey period according to publication data in the ACL Anthology, meaning we have survey responses from about 5% of the total." |
| | |
| | ==== Generation Lab 2023 ==== |
| | |
| | * [[https://www.generationlab.org/axios-generationlab-syracuse| |
| | AI EXPERT SURVEY (n=216 computer science professors)]] (Generation Lab, 2023) |
| | |
| | ==== Expert Survey on Progress in AI 2023 ==== |
| | |
| * [[https://wiki.aiimpacts.org/ai_timelines/predictions_of_human-level_ai_timelines/ai_timeline_surveys/2023_expert_survey_on_progress_in_ai|2023 Expert Survey on Progress in AI]] (Grace et al. 2023) | * [[https://wiki.aiimpacts.org/ai_timelines/predictions_of_human-level_ai_timelines/ai_timeline_surveys/2023_expert_survey_on_progress_in_ai|2023 Expert Survey on Progress in AI]] (Grace et al. 2023) |
| |
