AI Impacts Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== 2022 Expert Survey on Progress in AI ======

/* 


10/11/2023 Harlan: 
(You can now make comments on wiki using the syntax with the stars and slashes that you'll see below and above this comment. Readers will not be able to see this text.)
This wiki page is going to be hyperlinked in the survey invitation for the 2023 survey invites, so please be extra careful not to break it in the next few weeks!


*/
// Published 04 August, 2022; last updated 26 May, 2023 //



The 2022 Expert Survey on Progress in AI (2022 ESPAI) is a survey of machine learning researchers that AI Impacts ran in June-August 2022.






===== Background =====


<HTML>
<p>The 2022 ESPAI is a rerun of the <a href="/doku.php?id=ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:2016_expert_survey_on_progress_in_ai">2016 Expert Survey on Progress in AI</a> that researchers at AI Impacts previously collaborated on with others. Almost all of the questions were identical, and both surveyed authors who recently published in NeurIPS and ICML, major machine learning conferences.</p>
</HTML>


<HTML>
<p>Zhang et al ran a followup survey in 2019 (published in 2022)<span class="easy-footnote-margin-adjust" id="easy-footnote-1-3246"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-1-3246" title='Zhang, Baobao, Noemi Dreksler, Markus Anderljung, Lauren Kahn, Charlie Giattino, Allan Dafoe, and Michael Horowitz. “Forecasting AI Progress: Evidence from a Survey of Machine Learning Researchers,” June 8, 2022. &lt;a href="https://doi.org/10.48550/arXiv.2206.04132"&gt;https://doi.org/10.48550/arXiv.2206.04132&lt;/a&gt;.'><sup>1</sup></a></span> however they reworded or altered many questions, including the definitions of HLMI, so much of their data is not directly comparable to that of the 2016 or 2022 surveys, especially in light of large potential for framing effects observed.</p>
</HTML>


===== Methods =====


==== Population ====


<HTML>
<p>We contacted approximately 4271 researchers who published at the conferences NeurIPS or ICML in 2021. These people were selected by taking all of the authors at those conferences and randomly allocating them between this survey and a survey being run by others. We then contacted those whose email addresses we could find. We found email addresses in papers published at those conferences, in other public data, and in records from our previous survey and Zhang et al 2022. We received 738 responses, some partial, for a 17% response rate.</p>
</HTML>


<HTML>
<p>Participants who previously participated in the the 2016 ESPAI or Zhang et al surveys received slightly longer surveys, and received questions which they had received in past surveys (where random subsets of questions were given), rather than receiving newly randomized questions. This was so that they could also be included in a ‘matched panel’ survey, in which we contacted all researchers who completed the 2016 ESPAI or Zhang et al surveys, to compare responses from exactly the same samples of researchers over time. These surveys contained additional questions matching some of those in the Zhang et al survey.</p>
</HTML>


==== Contact ====


<HTML>
<p>We invited the selected researchers to take the survey via email. We accepted responses between June 12 and August 3, 2022.</p>
</HTML>


==== Questions ====


<HTML>
<p>The full list of survey questions is available below, as exported from the survey software. The export does not preserve pagination, or data about survey flow. Participants received randomized subsets of these questions, so the survey each person received was much shorter than that shown below.</p>
</HTML>


<HTML>
<div class="wp-block-file">
<object aria-label="Embed of 2022ESPAIV." class="wp-block-file__embed" data="https://aiimpacts.org/wp-content/uploads/2022/08/2022ESPAIV.pdf" style="width:100%;height:600px" type="application/pdf">
</object><a href="https://aiimpacts.org/wp-content/uploads/2022/08/2022ESPAIV.pdf" id="wp-block-file--media-e41f2945-7645-4eaa-8bde-58192430e913">2022ESPAIV</a><a aria-describedby="wp-block-file--media-e41f2945-7645-4eaa-8bde-58192430e913" class="wp-block-file__button" download="" href="https://aiimpacts.org/wp-content/uploads/2022/08/2022ESPAIV.pdf">Download</a>
</div>
</HTML>


<HTML>
<p>A small number of changes were made to questions since the 2016 survey (list forthcoming).</p>
</HTML>

==== Data cleaning ====

Edits were made to the raw data before analysis in the hope of preserving its intended meaning, including but not limited to:
  * Text such as '%' was removed from numerical answers
  * Some answers that were logically impossible in a way that implied misunderstanding of the question were ignored (e.g. when asked for the probability of an event happening by ascending numbers of years, the numbers should not be descending)
  * Some answers appeared designed to add to 100% in cases where that suggested a misunderstanding of the question. If a person sometimes had logically impossible answers which suggested misunderstanding the question, and sometimes had answers which were possible but also added to 100% suggesting the continuation of the same misunderstanding, we also excluded those answers.

===== Definitions =====


<HTML>
<p>‘HLMI’ was defined as follows:</p>
</HTML>


<HTML>
<p><em>The following questions ask about ‘high–level machine intelligence’ (HLMI). Say we have ‘high-level machine intelligence’ when unaided machines can accomplish every task better and more cheaply than human workers. Ignore aspects of tasks for which being a human is intrinsically advantageous, e.g. being accepted as a jury member. Think feasibility, not adoption.</em></p>
</HTML>


===== Results =====


==== Data ====


<HTML>
<p>The anonymized dataset is available <a href="https://docs.google.com/spreadsheets/d/1u_qcG6erXkH4EJgygl2fpkpJENAv6-kFWJejsw1oA1Q/edit?usp=sharing">here</a>.</p>
</HTML>


==== Summary of results ====


<HTML>
<ul>
<li><div class="li"><strong>The aggregate forecast time to a 50% chance of HLMI was 37 years, i.e. 2059</strong> (not including data from questions about the conceptually similar Full Automation of Labor, which in 2016 received much later estimates)<strong>.</strong> This timeline has become about eight years shorter in the six years since 2016, when the aggregate prediction put 50% probability at 2061, i.e. 45 years out. Note that these estimates are conditional on “human scientific activity continu[ing] without major negative disruption.”</div></li>
<li><div class="li"><strong>The median respondent believes the probability that the long-run effect of advanced AI on humanity will be “extremely bad (e.g., human extinction)” is 5%.</strong> This is the same as it was in 2016 (though Zhang et al 2022 found 2% in a similar but non-identical question). Many respondents were substantially more concerned: 48% of respondents gave at least 10% chance of an extremely bad outcome. But some much less concerned: 25% put it at 0%.</div></li>
<li><div class="li"><strong>The median respondent believes society should prioritize AI safety research “more” than it is currently prioritized.</strong> Respondents chose from “much less,” “less,” “about the same,” “more,” and “much more.” 69% of respondents chose “more” or “much more,” up from 49% in 2016.</div></li>
<li><div class="li"><strong>The median respondent thinks there is an “about even chance” that a stated argument for an intelligence explosion is broadly correct.</strong> 54% of respondents say the likelihood that it is correct is “about even,” “likely,” or “very likely” (corresponding to probability &gt;40%), similar to 51% of respondents in 2016. The median respondent also believes machine intelligence will probably (60%) be “vastly better than humans at all professions” within 30 years of HLMI, and the rate of global technological improvement will probably (80%) dramatically increase (e.g., by a factor of ten) as a result of machine intelligence within 30 years of HLMI.</div></li>
</ul>
</HTML>


==== High-level machine intelligence (HLMI) timelines ====

===Basic HLMI timelines===

<HTML>
<p>The aggregate forecast time to HLMI was 36.6 years, conditional on “human scientific activity continu[ing] without major negative disruption.” and considering only questions using the HLMI definition. We have not yet analyzed data about the conceptually similar Full Automation of Labor (FAOL), which in 2016 prompted much later timeline estimates. Thus this timeline figure is expected to be low relative to an overall estimate from this survey.</p>
</HTML>


<HTML>
<p>This aggregate is the 50th percentile date in an equal mixture of probability distributions created by fitting a gamma distribution to each person’s answers to three questions either about the probability of HLMI occurring by a given year or the year at which a given probability would obtain.</p>
</HTML>


<HTML>
<figure class="wp-block-image size-large is-resized">
<a href="http://aiimpacts.org/wp-content/uploads/2022/08/Screen-Shot-2022-08-04-at-02.55.28.jpg"><img alt="" class="wp-image-3248" height="480" sizes="(max-width: 512px) 100vw, 512px" src="https://aiimpacts.org/wp-content/uploads/2022/08/Screen-Shot-2022-08-04-at-02.55.28-1024x960.jpg" srcset="https://aiimpacts.org/wp-content/uploads/2022/08/Screen-Shot-2022-08-04-at-02.55.28-1024x960.jpg 1024w, https://aiimpacts.org/wp-content/uploads/2022/08/Screen-Shot-2022-08-04-at-02.55.28-300x281.jpg 300w, https://aiimpacts.org/wp-content/uploads/2022/08/Screen-Shot-2022-08-04-at-02.55.28-768x720.jpg 768w, https://aiimpacts.org/wp-content/uploads/2022/08/Screen-Shot-2022-08-04-at-02.55.28.jpg 1126w" width="512"/></a>
<figcaption>
                  Figure 1: Gamma distributions inferred for each individual.
                </figcaption>
</figure>
</HTML>


<HTML>
<figure class="wp-block-image size-full is-resized">
<a href="https://aiimpacts.org/wp-content/uploads/2022/08/2016_cdfs.jpg"><img alt="" class="wp-image-3249" height="420" loading="lazy" sizes="(max-width: 420px) 100vw, 420px" src="https://aiimpacts.org/wp-content/uploads/2022/08/2016_cdfs.jpg" srcset="https://aiimpacts.org/wp-content/uploads/2022/08/2016_cdfs.jpg 840w, https://aiimpacts.org/wp-content/uploads/2022/08/2016_cdfs-300x300.jpg 300w, https://aiimpacts.org/wp-content/uploads/2022/08/2016_cdfs-150x150.jpg 150w, https://aiimpacts.org/wp-content/uploads/2022/08/2016_cdfs-768x768.jpg 768w" width="420"/></a>
<figcaption>
                  Figure 2: Gamma distributions inferred for each individual, 2016 data
                </figcaption>
</figure>
</HTML>

===HLMI timelines via automation of labor===

Participants were either asked about the probability of an occupation being fully automated by a given year or asked about the year at which a given probability would obtain. We have not yet used the methodology in the above section to aggregate the results from the two questions into a single prediction. Below are the results for the fixed-year version of the question.

{{:ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:automationlikelihood.png|}}

==== Impacts of HLMI ====


== Question ==


<HTML>
<p>Participants were asked:</p>
</HTML>


<HTML>
<p>Assume for the purpose of this question that HLMI will at some point exist. How positive or negative do you expect the overall impact of this to be on humanity, in the long run? Please answer by saying how probable you find the following kinds of impact, with probabilities adding to 100%:</p>
</HTML>


<HTML>
<p>______ Extremely good (e.g. rapid growth in human flourishing) (1)</p>
</HTML>


<HTML>
<p>______ On balance good (2)</p>
</HTML>


<HTML>
<p>______ More or less neutral (3)</p>
</HTML>


<HTML>
<p>______ On balance bad (4)</p>
</HTML>


<HTML>
<p>______ Extremely bad (e.g. human extinction) (5)</p>
</HTML>


== Answers ==

**Medians**

<HTML>
<ul>
<li><div class="li">Extremely good: 10%</div></li>
<li><div class="li">On balance good: 20%</div></li>
<li><div class="li">More or less neutral: 15%</div></li>
<li><div class="li">On balance bad: 10%</div></li>
<li><div class="li">Extremely bad: 5%</div></li>
</ul>
</HTML>

Note that this means that at least half of respondents placed at least 5% probability on an extremely bad outcome.

**Means**

<HTML>
<ul>
<li><div class="li">Extremely good: 24%</div></li>
<li><div class="li">On balance good: 26%</div></li>
<li><div class="li">More or less neutral: 18%</div></li>
<li><div class="li">On balance bad: 17%</div></li>
<li><div class="li">Extremely bad: 14%</div></li>
</ul>
</HTML>

{{:ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:howbad-exploded.png?|}}

**Full distribution of responses**

{{:ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:hlmivalue2022final4.png?direct|}}

The areas allocated to different futures correspond to the means in the last subsection, so for instance 14% of the area is in extremely bad futures.

For comparison, here is the full distribution of responses from the [[ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:2016_expert_survey_on_progress_in_ai|2016 survey]]:

{{:ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:hlmivalue2016final4.png?direct|}}

Total area given to different scenarios in 2016 data:
  * Extremely good: 27%
  * On balance good: 30%
  * More or less neutral: 20%
  * On balance bad: 14%
  * Extremely bad: 9%

Here is the 2022 data again, ordered by overall optimism:

{{:ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:how_positive_or_negative_will_the_impacts_of_high-level_machine_intelligence_on_humanity_be_in_the_long_run_2022_14_.png?direct|}}

==== Intelligence explosion ====


=== Probability of dramatic technological speedup ===


== Question ==


<HTML>
<p>Participants were asked:</p>
</HTML>


<HTML>
<p>Assume that HLMI will exist at some point. How likely do you then think it is that the rate of global technological improvement will dramatically increase (e.g. by a factor of ten) as a result of machine intelligence:</p>
</HTML>


<HTML>
<p>Within <strong>two years</strong> of that point?       ___% chance</p>
</HTML>


<HTML>
<p>Within <strong>thirty years</strong> of that point?    ___% chance</p>
</HTML>


== Answers ==


<HTML>
<p>Median P(within <strong>two years</strong>) = 20% (20% in 2016)</p>
</HTML>


<HTML>
<p>Median P(within <strong>thirty years</strong>) = 80% (80% in 2016)</p>
</HTML>


=== Probability of superintelligence ===


== Question ==


<HTML>
<p>Participants were asked:</p>
</HTML>


<HTML>
<p>Assume that HLMI will exist at some point. How likely do you think it is that there will be machine intelligence that is <strong>vastly better</strong> than humans at all professions (i.e. that is vastly more capable or vastly cheaper):</p>
</HTML>


<HTML>
<p>Within <strong>two years</strong> of that point?       ___% chance</p>
</HTML>


<HTML>
<p>Within <strong>thirty years</strong> of that point?    ___% chance</p>
</HTML>


== Answers ==


<HTML>
<p>Median P(…within <strong>two years</strong>) = 10% (10% in 2016)</p>
</HTML>


<HTML>
<p>Median P(…within <strong>thirty years</strong>) = 60% (50% in 2016)</p>
</HTML>


=== Chance that the intelligence explosion argument is about right ===


== Question ==


<HTML>
<p>Participants were asked:</p>
</HTML>


<HTML>
<p>Some people have argued the following:</p>
</HTML>


<HTML>
<p><em>If AI systems do nearly all research and development, improvements in AI will accelerate the pace of technological progress, including further progress in AI.</em></p>
</HTML>


<HTML>
<p><em>Over a short period (less than 5 years), this feedback loop could cause technological progress to become more than an order of magnitude faster.</em></p>
</HTML>


<HTML>
<p>How likely do you find this argument to be broadly correct?</p>
</HTML>


<HTML>
<ul>
<li><div class="li">Quite unlikely (0-20%)</div></li>
<li><div class="li">Unlikely (21-40%)</div></li>
<li><div class="li">About even chance (41-60%)</div></li>
<li><div class="li">Likely (61-80%)</div></li>
<li><div class="li">Quite likely (81-100%)</div></li>
</ul>
</HTML>


== Answers ==


<HTML>
<ul>
<li><div class="li">20% quite unlikely (25% in 2016)</div></li>
<li><div class="li">26% unlikely (24% in 2016)</div></li>
<li><div class="li">21% about even chance (22% in 2016)</div></li>
<li><div class="li">26% likely (17% in 2016)</div></li>
<li><div class="li">7% quite likely (12% in 2016)</div></li>
</ul>
</HTML>

{{:ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:intelligenceexplosionhbar.png|}}

==== Causes of AI progress ====

Participants were asked about the sensitivity of progress in AI capabilities to various changes in
inputs.

== 50% less research effort ==

"Imagine that over the past decade, only half as much researcher effort had gone into
AI research. For instance, if there were actually 1,000 researchers, imagine that there
had been only 500 researchers (of the same quality).
How much less progress in AI capabilities would you expect to have seen?"

The median response was 25%.

== Less cheap hardware ==

"Over the last n years the cost of computing hardware has fallen by a factor of 20.
Imagine instead that the cost of computing hardware had fallen by only a factor of 5
over that time (around half as far on a log scale).
How much less progress in AI capabilities would you expect to have seen?"

The median response was 60%.

== 50% less work on training sets ==

"Imagine that over the past decade, there had only been half as much effort put into
increasing the size and availability of training datasets. For instance, perhaps there
are only half as many datasets, or perhaps existing datasets are substantially smaller or
lower quality.
How much less progress in AI capabilities would you expect to have seen?"

The median response was 50%.

== 50% less funding ==

"Imagine that over the past decade, AI research had half as much funding (in both
academic and industry labs). For instance, if the average lab had a budget of \$20 million
each year, suppose their budget had only been \$10 million each year.
How much less progress in AI capabilities would you expect to have seen?"

The median response was 35%.

== 50% less algorithmic progress ==

"Imagine that over the past decade, there had been half as much progress in AI
algorithms. You might imagine this as conceptual insights being half as frequent.
How much less progress in AI capabilities would you expect to have seen?"

The median response was 50%.

{{:ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:aiprogress.png|}}

==== Existential risk ====


<HTML>
<p>In an above question, participants’ credence in “extremely bad” outcomes of HLMI have median 5% and mean 14%. To better clarify what participants mean by this, we also asked a subset of participants one of the following questions, which did not appear in the 2016 survey:</p>
</HTML>


=== Extinction from AI ===


<HTML>
<p>Participants were asked:</p>
</HTML>


<HTML>
<p>What probability do you put on future AI advances causing human extinction or similarly permanent and severe disempowerment of the human species? </p>
</HTML>


== Answers ==


<HTML>
<p>Median 5%.</p>
</HTML>


=== Extinction from human failure to control AI ===


<HTML>
<p>Participants were asked:</p>
</HTML>


<HTML>
<p>What probability do you put on human inability to control future advanced AI systems causing human extinction or similarly permanent and severe disempowerment of the human species?</p>
</HTML>


== Answers ==


<HTML>
<p>Median 10%.</p>
</HTML>


<HTML>
<p>This question is more specific and thus necessarily less probable than the previous question, but it was given a higher probability at the median. This could be due to noise (different random subsets of respondents received the questions, so there is no logical requirement that their answers cohere), or due to the <a href="https://en.wikipedia.org/wiki/Representativeness_heuristic">representativeness heuristic</a>.</p>
</HTML>


==== Safety ====


=== General safety ===


== Question ==


<HTML>
<p>Participants were asked:</p>
</HTML>


<HTML>
<p>Let ‘<strong>AI safety research</strong>’ include any AI-related research that, rather than being primarily aimed at improving the <em>capabilities</em> of AI systems, is instead primarily aimed at <em>minimizing potential risks</em> of AI systems (beyond what is already accomplished for those goals by increasing AI system capabilities).</p>
</HTML>


<HTML>
<p>Examples of AI safety research might include:</p>
</HTML>


<HTML>
<ul>
<li><div class="li">Improving the human-interpretability of machine learning algorithms for the purpose of improving the safety and robustness of AI systems, not focused on improving AI capabilities</div></li>
<li><div class="li">Research on long-term existential risks from AI systems</div></li>
<li><div class="li">AI-specific formal verification research</div></li>
<li><div class="li">Policy research about how to maximize the public benefits of AI</div></li>
</ul>
</HTML>


<HTML>
<p>How much should society prioritize <strong>AI safety research</strong>, relative to how much it is currently prioritized?</p>
</HTML>


<HTML>
<ul>
<li><div class="li">Much less</div></li>
<li><div class="li">Less</div></li>
<li><div class="li">About the same</div></li>
<li><div class="li">More</div></li>
<li><div class="li">Much more</div></li>
</ul>
</HTML>


== Answers ==


<HTML>
<ul>
<li><div class="li">Much less: 2% (5% in 2016)</div></li>
<li><div class="li">Less: 9% (8% in 2016)</div></li>
<li><div class="li">About the same: 20% (38% in 2016)</div></li>
<li><div class="li">More: 35% (35% in 2016)</div></li>
<li><div class="li">Much more: 33% (14% in 2016)</div></li>
</ul>
</HTML>


<HTML>
<p>69% of respondents think society should prioritize AI safety research more or much more, up from 49% in 2016.</p>
</HTML>

{{:ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:how_much_should_society_prioritize_ai_safety_research_relative_to_how_much_it_is_currently_prioritized_1_.png?600|}}

=== Stuart Russell’s problem ===


== Question ==


<HTML>
<p>Participants were asked:</p>
</HTML>


<HTML>
<p>Stuart Russell summarizes an argument for why highly advanced AI might pose a risk as follows:</p>
</HTML>


<HTML>
<p><em>The primary concern [with highly advanced AI] is not spooky emergent consciousness but simply the ability to make high-quality decisions. Here, quality refers to the expected outcome utility of actions taken […]. Now we have a problem:</em></p>
</HTML>


<HTML>
<p><em>1. The utility function may not be perfectly aligned with the values of the human race, which are (at best) very difficult to pin down.</em></p>
</HTML>


<HTML>
<p><em>2. Any sufficiently capable intelligent system will prefer to ensure its own continued existence and to acquire physical and computational resources – not for their own sake, but to succeed in its assigned task.</em></p>
</HTML>


<HTML>
<p><em>A system that is optimizing a function of n variables, where the objective depends on a subset of size k&lt;n, will often set the remaining unconstrained variables to extreme values; if one of those unconstrained variables is actually something we care about, the solution found may be highly undesirable.  This is essentially the old story of the genie in the lamp, or the sorcerer’s apprentice, or King Midas: you get exactly what you ask for, not what you want.</em></p>
</HTML>


<HTML>
<p>Do you think this argument points at an important problem?</p>
</HTML>


<HTML>
<ul>
<li><div class="li">No, not a real problem.</div></li>
<li><div class="li">No, not an important problem.</div></li>
<li><div class="li">Yes, a moderately important problem.</div></li>
<li><div class="li">Yes, a very important problem.</div></li>
<li><div class="li">Yes, among the most important problems in the field.</div></li>
</ul>
</HTML>


<HTML>
<p>How valuable is it to work on this problem <strong><em>today</em></strong>, compared to other problems in AI?</p>
</HTML>


<HTML>
<ul>
<li><div class="li">Much less valuable</div></li>
<li><div class="li">Less valuable</div></li>
<li><div class="li">As valuable as other problems</div></li>
<li><div class="li">More valuable</div></li>
<li><div class="li">Much more valuable</div></li>
</ul>
</HTML>


<HTML>
<p>How hard do you think this problem is compared to other problems in AI?</p>
</HTML>


<HTML>
<ul>
<li><div class="li">Much easier</div></li>
<li><div class="li">Easier</div></li>
<li><div class="li">As hard as other problems</div></li>
<li><div class="li">Harder</div></li>
<li><div class="li">Much harder</div></li>
</ul>
</HTML>


== Answers ==


<HTML>
<p>Importance:</p>
</HTML>


<HTML>
<ul>
<li><div class="li">No, not a real problem: 4%</div></li>
<li><div class="li">No, not an important problem: 14%</div></li>
<li><div class="li">Yes, a moderately important problem: 24%</div></li>
<li><div class="li">Yes, a very important problem: 37%</div></li>
<li><div class="li">Yes, among the most important problems in the field: 21%</div></li>
</ul>
</HTML>


<HTML>
<p>Value today:</p>
</HTML>


<HTML>
<ul>
<li><div class="li">Much less valuable: 10%</div></li>
<li><div class="li">Less valuable: 30%</div></li>
<li><div class="li">As valuable as other problems: 33%</div></li>
<li><div class="li">More valuable: 19%</div></li>
<li><div class="li">Much more valuable: 8%</div></li>
</ul>
</HTML>


<HTML>
<p>Hardness:</p>
</HTML>


<HTML>
<ul>
<li><div class="li">Much easier: 5%</div></li>
<li><div class="li">Easier: 9%</div></li>
<li><div class="li">As hard as other problems: 29%</div></li>
<li><div class="li">Harder: 31%</div></li>
<li><div class="li">Much harder: 26%</div></li>
</ul>
</HTML>

===== Notable citations of 2022 Expert Survey on Progress in AI =====

Places where the [[ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:2022_expert_survey_on_progress_in_ai|2022 ESPAI]] has been cited:

  - Holt, Lester. 2023. "[[https://youtu.be/qRLrE2tkr2Y|AI ‘race to recklessness’ could have dire consequences, tech experts warn in new interview]]" //NBC Nightly News//
  - Feldman, Noah. 2023. "[[https://www.bloomberg.com/opinion/articles/2023-04-02/regulating-ai-might-require-a-new-federal-agency|Regulating AI Will Be Essential. And Complicated.]]" //Bloomberg//
  - Wallace-Wells, David. 2023. "[[https://www.nytimes.com/2023/03/27/opinion/ai-chatgpt-chatbots.html|A.I. Is Being Built by People Who Think It Might Destroy Us]]" //The New York Times//
  - Roser, Max. 2023. "[[https://ourworldindata.org/ai-timelines|AI timelines: What do experts in artificial intelligence expect for the future?]]" //Our World in Data//
  - Klein, Ezra. 2023. "[[https://www.nytimes.com/2023/03/21/podcasts/ezra-klein-podcast-transcript-kelsey-piper.html|Transcript: Ezra Klein Interviews Kelsey Piper]]" //The New York Times//
  - Klein, Ezra. 2023. "[[https://www.nytimes.com/2023/03/12/opinion/chatbots-artificial-intelligence-future-weirdness.html|This Changes Everything]]" //The New York Times//
  - Samuel, Sigal. 2023. "[[https://www.vox.com/the-highlight/23621198/artificial-intelligence-chatgpt-openai-existential-risk-china-ai-safety-technology|The case for slowing down AI"]] //Vox//
  - Lopez, German. 2023. "[[https://www.nytimes.com/2023/04/21/briefing/ai-chatgpt.html|Using A.I. in Everyday Life]]" //The New York Times//
  - 2023. "[[https://www.economist.com/science-and-technology/2023/04/19/how-generative-models-could-go-wrong|How generative models could go wrong]]" //The Economist//
  - Harari, Yuval, et al. 2023. "[[https://www.nytimes.com/2023/03/24/opinion/yuval-harari-ai-chatgpt.html|You Can Have the Blue Pill or the Red Pill, and We’re Out of Blue Pills]]" //The New York Times//
  - Hammond, Samuel. 2023. "[[https://www.politico.com/news/magazine/2023/05/08/manhattan-project-for-ai-safety-00095779|We Need a Manhattan Project for AI Safety]]" //Politico//
  - Nay, John. 2022. "[[https://law.stanford.edu/2022/09/25/aligning-ai-with-humans-by-leveraging-law-as-data/|Aligning AI with Humans by Leveraging Law as Data]]" //Center for Legal Informatics, Stanford University//

===== Contributions =====


<HTML>
<p>The survey was run by Katja Grace and Ben Weinstein-Raun. Data analysis was done by Zach Stein-Perlman, Ben Weinstein-Raun and John Salvatier. This page was written by Zach Stein-Perlman and Katja Grace.</p>
</HTML>


<HTML>
<p>We thank many colleagues and friends for help, discussion and encouragement, including John Salvatier, Nick Beckstead, Howie Lempel, Joe Carlsmith, Leopold Aschenbrenner, Ramana Kumar, Jimmy Rintjema, Jacob Hilton, Ajeya Cotra, Scott Siskind, Chana Messinger, Noemi Dreksler, and Baobao Zhang.</p>
</HTML>


<HTML>
<p>We also thank the expert participants who spent time sharing their impressions with us, including:</p>
</HTML>


<HTML>
<p>Michał Zając<br/>
              Morten Goodwin<br/>
              Yue Sun<br/>
              Ningyuan Chen<br/>
              Egor Kostylev<br/>
              Richard Antonello<br/>
              Elia Turner<br/>
              Andrew C Li<br/>
              Zachary Markovich<br/>
              Valentina Zantedeschi<br/>
              Michael Cooper<br/>
              Thomas A Keller<br/>
              Marc Cavazza<br/>
              Richard Vidal<br/>
              David Lindner<br/>
              Xuechen (Chen) Li<br/>
              Alex M. Lamb<br/>
              Tristan Aumentado-Armstrong<br/>
              Ferdinando Fioretto<br/>
              Alain Rossier<br/>
              Wentao Zhang<br/>
              Varun Jampani<br/>
              Derek Lim<br/>
              Muchen Li<br/>
              Cong Hao<br/>
              Yao-Yuan Yang<br/>
              Linyi Li<br/>
              Stéphane D’Ascoli<br/>
              Lang Huang<br/>
              Maxim Kodryan<br/>
              Hao Bian<br/>
              Orestis Paraskevas<br/>
              David Madras<br/>
              Tommy Tang<br/>
              Li Sun<br/>
              Stefano V Albrecht<br/>
              Tristan Karch<br/>
              Muhammad A Rahman<br/>
              Runtian Zhai<br/>
              Benjamin Black<br/>
              Karan Singhal<br/>
              Lin Gao<br/>
              Ethan Brooks<br/>
              Cesar Ferri<br/>
              Dylan Campbell<br/>
              Xujiang Zhao<br/>
              Jack Parker-Holder<br/>
              Michael Norrish<br/>
              Jonathan Uesato<br/>
              Yang An<br/>
              Maheshakya Wijewardena<br/>
              Ulrich Neumann<br/>
              Lucile Ter-Minassian<br/>
              Alexander Matt turner<br/>
              Subhabrata Dutta<br/>
              Yu-Xiang Wang<br/>
              Yao Zhang<br/>
              Joanna Hong<br/>
              Yao Fu<br/>
              Wenqing Zheng<br/>
              Louis C Tiao<br/>
              Hajime Asama<br/>
              Chengchun Shi<br/>
              Moira R Dillon<br/>
              Yisong Yue<br/>
              Aurélien Bellet<br/>
              Yin Cui<br/>
              Gang Hua<br/>
              Jongheon Jeong<br/>
              Martin Klissarov<br/>
              Aran Nayebi<br/>
              Fabio Maria Carlucci<br/>
              Chao Ma<br/>
              Sébastien Gambs<br/>
              Rasoul Mirzaiezadeh<br/>
              Xudong Shen<br/>
              Julian Schrittwieser<br/>
              Adhyyan Narang<br/>
              Fuxin Li<br/>
              Linxi Fan<br/>
              Johannes Gasteiger<br/>
              Karthik Abinav Sankararaman<br/>
              Patrick Mineault<br/>
              Akhilesh Gotmare<br/>
              Jibang Wu<br/>
              Mikel Landajuela<br/>
              Jinglin Liu<br/>
              Qinghua Hu<br/>
              Noah Siegel<br/>
              Ashkan Khakzar<br/>
              Nathan Grinsztajn<br/>
              Julian Lienen<br/>
              Xiaoteng Ma<br/>
              Mohamad H Danesh<br/>
              Ke ZHANG<br/>
              Feiyu Xiong<br/>
              Wonjae Kim<br/>
              Michael Arbel<br/>
              Piotr Skowron<br/>
              Lê-Nguyên Hoang<br/>
              Travers Rhodes<br/>
              Liu Ziyin<br/>
              Hossein Azizpour<br/>
              Karl Tuyls<br/>
              Hangyu Mao<br/>
              Yi Ma<br/>
              Junyi Li<br/>
              Yong Cheng<br/>
              Aditya Bhaskara<br/>
              Xia Li<br/>
              Danijar Hafner<br/>
              Brian Quanz<br/>
              Fangzhou Luo<br/>
              Luca Cosmo<br/>
              Scott Fujimoto<br/>
              Santu Rana<br/>
              Michael Curry<br/>
              Karol Hausman<br/>
              Luyao Yuan<br/>
              Samarth Sinha<br/>
              Matthew McLeod<br/>
              Hao Shen<br/>
              Navid Naderializadeh<br/>
              Alessio Micheli<br/>
              Zhenbang You<br/>
              Van Huy Vo<br/>
              Chenyang Wu<br/>
              Thanard Kurutach<br/>
              Vincent Conitzer<br/>
              Chuang Gan<br/>
              Chirag Gupta<br/>
              Andreas Schlaginhaufen<br/>
              Ruben Ohana<br/>
              Luming Liang<br/>
              Marco Fumero<br/>
              Paul Muller<br/>
              Hana Chockler<br/>
              Ming Zhong<br/>
              Jiamou Liu<br/>
              Sumeet Agarwal<br/>
              Eric Winsor<br/>
              Ruimeng Hu<br/>
              Changjian Shui<br/>
              Yiwei Wang<br/>
              Joey Tianyi Zhou<br/>
              Anthony L. Caterini<br/>
              Guillermo Ortiz-Jimenez<br/>
              Iou-Jen Liu<br/>
              Jiaming Liu<br/>
              Michael Perlmutter<br/>
              Anurag Arnab<br/>
              Ziwei Xu<br/>
              John Co-Reyes<br/>
              Aravind Rajeswaran<br/>
              Roy Fox<br/>
              Yong-Lu Li<br/>
              Carl Yang<br/>
              Divyansh Garg<br/>
              Amit Dhurandhar<br/>
              Harris Chan<br/>
              Tobias Schmidt<br/>
              Robi Bhattacharjee<br/>
              Marco Nadai<br/>
              Reid McIlroy-Young<br/>
              Wooseok Ha<br/>
              Jesse Mu<br/>
              Neale Ratzlaff<br/>
              Kenneth Borup<br/>
              Binghong Chen<br/>
              Vikas Verma<br/>
              Walter Gerych<br/>
              Shachar Lovett<br/>
              Zhengyu Zhao<br/>
              Chandramouli Chandrasekaran<br/>
              Richard Higgins<br/>
              Nicholas Rhinehart<br/>
              Blaise Agüera Y Arcas<br/>
              Santiago Zanella-Beguelin<br/>
              Dian Jin<br/>
              Scott Niekum<br/>
              Colin A. Raffel<br/>
              Sebastian Goldt<br/>
              Yali Du<br/>
              Bernardo Subercaseaux<br/>
              Hui Wu<br/>
              Vincent Mallet<br/>
              Ozan Özdenizci<br/>
              Timothy Hospedales<br/>
              Lingjiong Zhu<br/>
              Cheng Soon Ong<br/>
              Shahab Bakhtiari<br/>
              Huan Zhang<br/>
              Banghua Zhu<br/>
              Byungjun Lee<br/>
              Zhenyu Liao<br/>
              Adrien Ecoffet<br/>
              Vinay Ramasesh<br/>
              Jesse Zhang<br/>
              Soumik Sarkar<br/>
              Nandan Kumar Jha<br/>
              Daniel S Brown<br/>
              Neev Parikh<br/>
              Chen-Yu Wei<br/>
              David K. Duvenaud<br/>
              Felix Petersen<br/>
              Songhua Wu<br/>
              Huazhu Fu<br/>
              Roger B Grosse<br/>
              Matteo Papini<br/>
              Peter Kairouz<br/>
              Burak Varici<br/>
              Fabio Roli<br/>
              Mohammad Zalbagi Darestani<br/>
              Jiamin He<br/>
              Lys Sanz Moreta<br/>
              Xu-Hui Liu<br/>
              Qianchuan Zhao<br/>
              Yulia Gel<br/>
              Jan Drgona<br/>
              Sajad Khodadadian<br/>
              Takeshi Teshima<br/>
              Igor T Podolak<br/>
              Naoya Takeishi<br/>
              Man Shun Ang<br/>
              Mingli Song<br/>
              Jakub Tomczak<br/>
              Lukasz Szpruch<br/>
              Micah Goldblum<br/>
              Graham W. Taylor<br/>
              Tomasz Korbak<br/>
              Maheswaran Sathiamoorthy<br/>
              Lan-Zhe Guo<br/>
              Simone Fioravanti<br/>
              Lei Jiao<br/>
              Davin Choo<br/>
              Kristy Choi<br/>
              Varun Nair<br/>
              Rayana Jaafar<br/>
              Amy Greenwald<br/>
              Martin V. Butz<br/>
              Aleksey Tikhonov<br/>
              Samuel Gruffaz<br/>
              Yash Savani<br/>
              Rui Chen<br/>
              Ke Sun</p>
</HTML>



We thank FTX Future Fund for encouraging this project, though they did not ultimately fund it as anticipated due to the [[https://en.wikipedia.org/wiki/Bankruptcy_of_FTX|Bankruptcy of FTX]].


==== Suggested citation ====


<HTML>
<p>Katja Grace, Zach Stein-Perlman, Benjamin Weinstein-Raun, and John Salvatier, “2022 Expert Survey on Progress in AI.” <em>AI Impacts</em>, 3 Aug. 2022. <a href="/doku.php?id=ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:2022_expert_survey_on_progress_in_ai">https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/</a>.</p>
</HTML>


====== Notes ======


<HTML>
<ol class="easy-footnotes-wrapper">
<li><div class="li">
<span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-1-3246"></span>Zhang, Baobao, Noemi Dreksler, Markus Anderljung, Lauren Kahn, Charlie Giattino, Allan Dafoe, and Michael Horowitz. “Forecasting AI Progress: Evidence from a Survey of Machine Learning Researchers,” June 8, 2022. <a href="https://doi.org/10.48550/arXiv.2206.04132">https://doi.org/10.48550/arXiv.2206.04132</a>.<a class="easy-footnote-to-top" href="#easy-footnote-1-3246"></a>
</div></li>
</ol>
</HTML>
AI Impacts Wiki

User Tools

Site Tools

Page Tools
