This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:2023_expert_survey_on_progress_in_ai [2023/10/04 22:48] harlanstewart |
ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:2023_expert_survey_on_progress_in_ai [2024/01/30 01:17] (current) harlanstewart |
||
---|---|---|---|
Line 1: | Line 1: | ||
====== 2023 Expert Survey on Progress in AI ====== | ====== 2023 Expert Survey on Progress in AI ====== | ||
- | // Published 17 August, 2023 // | + | // Published 17 August, 2023. Last updated 29 January, 2024. // |
+ | The 2023 Expert Survey on Progress in AI is a survey of 2,778 AI researchers that AI Impacts ran in October 2023. | ||
+ | ===== Details ===== | ||
- | //The 2023 Expert Survey on Progress in AI is currently in progress, and this page will be updated as details and results become available.// | + | ==== Background ==== |
+ | The 2023 Expert Survey on Progress in AI (2023 ESPAI) is a rerun of the [[ai_timelines: | ||
- | AI Impacts is preparing to run the 2023 Expert Survey on Progress in AI (2023 ESPAI). | + | A preprint about the 2023 ESPAI is available [[https:// |
- | ===== Details ===== | + | ==== Survey methods |
- | ==== Background ==== | + | === Questions |
- | The 2023 ESPAI will be a rerun of the [[ai_timelines: | + | The questions in the 2023 ESPAI are nearly identical to those in the [[ai_timelines: |
- | ==== Survey | + | Some questions were added to the 2023 ESPAI which were not in either of the two previous surveys. We refined the new questions through an iterative process involving several rounds of testing the questions in verbal interviews with computer science graduate students and others. |
+ | |||
+ | {{ : | ||
+ | |||
+ | === Participants | ||
+ | |||
+ | We collected the names of authors who published in 2022 at a selection of top-tier machine learning conferences (NeurIPS, ICML, ICLR, AAAI, JMLR, and IJCAI), and were able to find email addresses for 20,066 (92%) of them. The resulting list of emails was put into a random order based on a random unique number assigned using Google Sheets' | ||
+ | |||
+ | The pilot study took place from October 11 to October 15 in 2023. Based on the response rates in the paid group versus the unpaid group, we decided to offer payment to all survey participants. A $50 reward will be issued through a third-party service. Depending on a participant' | ||
+ | |||
+ | On October 15, the survey was sent to the main survey group. The survey remained open until October 24, 2023. Out of the 20,066 emails we contacted, 1,607 (8%) bounced or failed, leaving 18,459 functioning email addresses. We received 2,778 responses, for a response rate of 15%. 95% of these responses were deemed ‘finished’ by Qualtrics. | ||
+ | |||
+ | === Changes from 2016 and 2022 ESPAI surveys === | ||
+ | |||
+ | These are some notable differences from the [[ai_timelines: | ||
+ | * We recruited participants from twice as many conferences as in 2022. | ||
+ | * We made some changes to the order and flow of the questions. | ||
+ | * We gave some participants a new version of the extinction risk question that was phrased to include a timeframe (" | ||
+ | * Some participants were given new questions that were not in the 2022 ESPAI. | ||
+ | |||
+ | Full data on differences between surveys is available [[https:// | ||
+ | |||
+ | ==== Definitions ==== | ||
+ | * **" | ||
+ | * **" | ||
+ | * **" | ||
+ | * **" | ||
+ | * **" | ||
+ | |||
+ | ==== Results ==== | ||
+ | The full dataset of anonymized responses to the survey is available [[https:// | ||
+ | |||
+ | ===Timing of human-level performance=== | ||
+ | We asked about the timing of human-level performance by asking some participants about how soon they expect " | ||
+ | ==High-level machine intelligence (HLMI)== | ||
+ | >Say we have ‘high-level machine intelligence’ when unaided machines can accomplish every task better and more cheaply than human workers. Ignore aspects of tasks for which being a human is intrinsically advantageous, | ||
+ | >For the purposes of this question, assume that human scientific activity continues without major negative disruption. | ||
+ | |||
+ | Half of the participants were asked about the number of years until they expected a 10%, 50%, and 90% probability of HLMI existing (fixed-probabilities framing) and half were asked about the probability they would give to HLMI existing in 10, 20, and 40 years (fixed-years framing). | ||
+ | |||
+ | We aggregated the 1714 responses to this question by fitting each response to a gamma CDF and finding the mean curve of those CDFs. The resulting aggregate forecast gives a 50% chance of HLMI by 2047, down thirteen years from 2060 in the 2022 ESPAI. | ||
+ | |||
+ | [{{: | ||
+ | |||
+ | ==Full automation of labor (FAOL)== | ||
+ | |||
+ | >Say we have reached ‘full automation of labor’ when all occupations are fully automatable. That is, when for any occupation, machines could be built to carry out the task better and more cheaply than human workers. | ||
+ | |||
+ | As with the HLMI question, half of the participants who were given the FAOL question were asked using the fixed-probabilities framing and the other half was asked using the fixed-years framing. | ||
+ | |||
+ | The 774 responses to this question were used to create an aggregate forecast which gives a 50% chance of FAOL by 2116, down 48 years from 2164 in the 2022 ESPAI. | ||
+ | |||
+ | [{{: | ||
+ | |||
+ | ===Intelligence explosion=== | ||
+ | We asked about the likelihood of an ' | ||
+ | ==Chance that the intelligence explosion argument is about right== | ||
+ | >Some people have argued the following: | ||
+ | >//If AI systems do nearly all research and development, | ||
+ | >//Over a short period (less than 5 years), this feedback loop could cause technological progress to become more than an order of magnitude faster.// | ||
+ | >How likely do you find this argument to be broadly correct? | ||
+ | |||
+ | Of the 299 responses to this question, | ||
+ | |||
+ | * 9% said "Quite likely (81-100%)" | ||
+ | * 20% said " | ||
+ | * 24% said "About even chance (41-60%)" | ||
+ | * 24% said " | ||
+ | * 23% said "Quite unlikely (0-20%)" | ||
+ | |||
+ | ==Probability of dramatic technological speedup== | ||
+ | >Assume that HLMI will exist at some point. | ||
+ | >How likely do you then think it is that the rate of global technological improvement will dramatically increase (e.g. by a factor of ten) as a result of machine intelligence: | ||
+ | >Within two years of that point? _% chance | ||
+ | >Within thirty years of that point? _% chance | ||
+ | |||
+ | There were 298 responses to this question. The median answer was 20% for two years and 80% for thirty years. | ||
+ | |||
+ | ==Probability of superintelligence== | ||
+ | >Assume that HLMI will exist at some point. | ||
+ | >How likely do you think it is that there will be machine intelligence that is vastly better than humans at all professions (i.e. that is vastly more capable or vastly cheaper): | ||
+ | >Within two years of that point? _% chance | ||
+ | >Within thirty years of that point? _% chance | ||
+ | |||
+ | There were 282 responses to this question. The median answer was 10% for two years and 60% for thirty years. | ||
+ | |||
+ | ===AI Interpretability in 2028=== | ||
+ | >For typical state-of-the-art AI systems in 2028, do you think it will be possible for users to know the true reasons for systems making a particular choice? By “true reasons” we mean the AI correctly explains its internal decision-making process in a way humans can understand. By “true reasons” we do not mean the decision itself is correct. | ||
+ | |||
+ | Of the 912 responses to this question, | ||
+ | * 5% said "Very unlikely (< | ||
+ | * 35% said " | ||
+ | * 20% said "Even odds (40-60%)" | ||
+ | * 15% said " | ||
+ | * 5% said "Very likely (> | ||
+ | |||
+ | ===How concerning are 11 future AI-related scenarios? | ||
+ | 1345 participants rated their level of concern for 11 AI-related scenarios over the next thirty years. As measured by the percentage of respondents who thought a scenario constituted either a “substantial” or “extreme” concern, the scenarios worthy of most concern were: spread of false information e.g. deepfakes (86%), manipulation of large-scale public opinion trends (79%), AI letting dangerous groups make powerful tools (e.g. engineered viruses) (73%), authoritarian rulers using AI to control their populations (73%), and AI systems worsening economic inequality by disproportionately benefiting certain individuals (71%). | ||
+ | |||
+ | {{: | ||
+ | |||
+ | ===Overall impact of HLMI=== | ||
+ | >Assume for the purpose of this question that HLMI will at some point exist. How positive or negative do you expect the overall impact of this to be on humanity, in the long run? | ||
+ | |||
+ | Participants were asked for the probability they would give to different kinds of impact. Below are the medians and means of the 2704 responses: | ||
+ | |||
+ | ^ Overall impact of HLMI ^ Median response ^ Mean response ^ | ||
+ | | " | ||
+ | | "On balance good" | 25% | 29% | | ||
+ | | "More or less neutral" | ||
+ | | "On balance bad" | 15% | 18% | | ||
+ | | " | ||
+ | |||
+ | [{{: | ||
+ | |||
+ | ===Preferred rate of progress=== | ||
+ | >What rate of global AI progress over the next five years would make you feel most optimistic for humanity' | ||
+ | |||
+ | Of the 675 responses to this question, | ||
+ | |||
+ | * 4.8% said "Much slower" | ||
+ | * 29.9% said " | ||
+ | * 26.9% said " | ||
+ | * 22.8% said " | ||
+ | * 15.6% said "Much faster" | ||
+ | |||
+ | ===How soon will 39 tasks be feasible for AI?=== | ||
+ | Participants were asked about when each of 39 tasks would become " | ||
+ | would choose to.” Each respondent was asked about four of the tasks, so each task received around 250 estimates. | ||
+ | |||
+ | As with the questions about timelines to human performance, | ||
+ | |||
+ | [{{: | ||
+ | 50% chance of being met is represented by solid circles, open circles, and solid squares for tasks, occupations, | ||
+ | general human-level performance respectively.}}] | ||
+ | |||
+ | ===The alignment problem=== | ||
+ | >Stuart Russell summarizes an argument for why highly advanced AI might pose a risk as follows: | ||
+ | > | ||
+ | >//The primary concern [with highly advanced AI] is not spooky emergent consciousness but simply the ability to make high-quality decisions. Here, quality refers to the expected outcome utility of actions taken […]. Now we have a problem:// | ||
+ | > | ||
+ | >//1. The utility function may not be perfectly aligned with the values of the human race, which are (at best) very difficult to pin down.// | ||
+ | >//2. Any sufficiently capable intelligent system will prefer to ensure its own continued existence and to acquire physical and computational resources – not for their own sake, but to succeed in its assigned task. | ||
+ | // | ||
+ | > | ||
+ | >//A system that is optimizing a function of n variables, where the objective depends on a subset of size k<n, will often set the remaining unconstrained variables to extreme values; if one of those unconstrained variables is actually something we care about, the solution found may be highly undesirable. | ||
+ | |||
+ | =="Do you think this argument points at an important problem?" | ||
+ | |||
+ | Of the 1322 who answered this question, | ||
+ | * 5% said "No, not a real problem" | ||
+ | * 9% said "No, not an important problem" | ||
+ | * 32% said "Yes, a moderately important problem" | ||
+ | * 41% said "Yes, a very important problem" | ||
+ | * 13% said "Yes, among the most important problems in the field" | ||
+ | |||
+ | =="How valuable is it to work on this problem today, compared to other problems in AI?" | ||
+ | |||
+ | Of the 1321 who answered this question, | ||
+ | * 9% said "Much more valuable" | ||
+ | * 22% said "More valuable" | ||
+ | * 39% said "As valuable" | ||
+ | * 22% said "Less valuable" | ||
+ | * 8% said "Much less valuable" | ||
+ | |||
+ | =="How hard do you think this problem is compared to other problems in AI?" | ||
+ | |||
+ | Of the 1274 who answered this question, | ||
+ | * 21% said "Much harder" | ||
+ | * 36% said " | ||
+ | * 30% said "As hard" | ||
+ | * 10% said " | ||
+ | * 3% said "Much easier" | ||
+ | |||
+ | ===How much should society prioritize AI safety research? | ||
+ | >Let 'AI safety research' | ||
+ | > | ||
+ | > * Improving the human-interpretability of machine learning algorithms for the purpose of improving the safety and robustness of AI systems, not focused on improving AI capabilities | ||
+ | > * Research on long-term existential risks from AI systems | ||
+ | > * AI-specific formal verification research | ||
+ | > * Developing methodologies to identify, measure, and mitigate biases in AI models to ensure fair and ethical decision-making. | ||
+ | > * Policy research about how to maximize the public benefits of AI | ||
+ | > | ||
+ | >How much should society prioritize AI safety research, relative to | ||
+ | how much it is currently prioritized? | ||
+ | |||
+ | Of the 1329 responses to this question, | ||
+ | * 2% said "Much less" | ||
+ | * 6% said " | ||
+ | * 22% said "About the same" | ||
+ | * 36% said " | ||
+ | * 34% said "Much more" | ||
+ | |||
+ | [{{: | ||
+ | |||
+ | ===Human extinction=== | ||
+ | We asked about the likelihood that AI will cause human extinction using three differently phrased questions, for which we collected 655-1321 responses each. | ||
+ | |||
+ | ^ Question phrasing ^ Median response ^ Mean response ^ | ||
+ | | "What probability do you put on future AI advances causing human extinction or similarly permanent and severe disempowerment of the human species?" | ||
+ | | "What probability do you put on human inability to control future advanced AI systems causing human extinction or similarly permanent and severe disempowerment of the human species?" | ||
+ | |"What probability do you put on future AI advances causing human extinction or similarly permanent and severe disempowerment of the human species within the next 100 years?" | ||
+ | |||
+ | {{: | ||
+ | |||
+ | ====Frequently asked questions==== | ||
+ | |||
+ | ===How does the seniority of the participants affect the results?=== | ||
- | For all authors who published at a selection of top-tier machine learning conferences in 2022 (including ICML, NeurIPS, | + | One reasonable concern about an expert survey might be that more senior experts are busier |
- | As in the previous expert surveys conducted by AI Impacts, different phrasings of some questions will be randomly assigned | + | ^Group^% who gave at least 5% odds to " |
+ | |All participants|57.80%| | ||
+ | |Has 100+ citations|62.30%| | ||
+ | |Has 1000+ citations|59.00%| | ||
+ | |Has 10,000+ citations|56.30%| | ||
+ | |Started PhD by 2018|58.80%| | ||
+ | |Started PhD by 2013|58.50%| | ||
+ | |Started PhD by 2003|54.70%| | ||
+ | |In current field 5+ years|54.40%| | ||
+ | |In current field 10+ years|51.40%| | ||
+ | |In current field 20+ years|48.00%| | ||
- | === Changes from past survey | + | ====Contributions==== |
+ | Authors of the 2023 Expert Survey on Progress in AI are Katja Grace, Julia Fabienne Sandkühler, | ||
- | These things are expected | + | Many thanks for help with this research |
- | * Participants from more venues | + | |
- | * Order of questions (demographics moved to the start) | + | |
- | * Addition of extinction risk question with timeframe | + |