|
|
|
|
— |
ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:2016_expert_survey_on_progress_in_ai [2022/09/21 07:37] (current)
|
| | ====== 2016 Expert Survey on Progress in AI ====== |
| | |
| | // Published 14 December, 2016; last updated 26 March, 2021 // |
| | |
| | |
| | |
| | <HTML> |
| | <p>The 2016 Expert Survey on Progress in AI is a survey of machine learning researchers that Katja Grace and John Salvatier of AI Impacts ran in collaboration with Allan Dafoe, Baobao Zhang, and Owain Evans in 2016.</p> |
| | </HTML> |
| | |
| | |
| | |
| | ===== Details ===== |
| | |
| | |
| | <HTML> |
| | <p>Some survey results are reported in <em><a href="https://arxiv.org/abs/1705.08807">When Will AI Exceed Human Performance? Evidence from AI Experts</a></em>. This page reports on results from those questions more fully, and results from some questions not included in the paper.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>The full list of survey questions is available <a href="/doku.php?id=ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:2016_espai_questions_printout">here</a> (<a href="https://www.dropbox.com/s/99os4grxlhf744m/Final%202016%20Expert%20Survey%20on%20Progress%20in%20AI.pdf?dl=0">pdf</a>). Participants received randomized subsets of these questions.</p> |
| | </HTML> |
| | |
| | |
| | ==== Definitions ==== |
| | |
| | |
| | <HTML> |
| | <p>Throughout the survey, ‘HLMI’ was defined as follows:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <blockquote> |
| | <p>The following questions ask about ‘high–level machine intelligence’ (HLMI). Say we have ‘high-level machine intelligence’ when unaided machines can accomplish every task better and more cheaply than human workers. Ignore aspects of tasks for which being a human is intrinsically advantageous, e.g. being accepted as a jury member. Think feasibility, not adoption.</p> |
| | </blockquote> |
| | </HTML> |
| | |
| | |
| | ==== Summary of results ==== |
| | |
| | |
| | <HTML> |
| | <p>Below is a table of summary results from the paper (the paper contains more results).</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <figure aria-describedby="caption-attachment-843" class="wp-caption alignnone" id="attachment_843" style="width: 829px"> |
| | <a href="http://aiimpacts.org/wp-content/uploads/2016/12/survey-summary.jpeg"><img alt="" class="wp-image-843 size-large" height="1024" sizes="(max-width: 829px) 100vw, 829px" src="https://aiimpacts.org/wp-content/uploads/2016/12/survey-summary-829x1024.jpeg" srcset="https://aiimpacts.org/wp-content/uploads/2016/12/survey-summary-829x1024.jpeg 829w, https://aiimpacts.org/wp-content/uploads/2016/12/survey-summary-243x300.jpeg 243w, https://aiimpacts.org/wp-content/uploads/2016/12/survey-summary-768x949.jpeg 768w, https://aiimpacts.org/wp-content/uploads/2016/12/survey-summary.jpeg 1128w" width="829"/></a> |
| | <figcaption class="wp-caption-text" id="caption-attachment-843"> |
| | <em><strong>Table S4 in <a href="https://arxiv.org/abs/1705.08807">Grace et al 2017</a>.</strong></em> |
| | </figcaption> |
| | </figure> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p><span class="name" data-wfid="c74b02a3a538">Some key interesting results, from our <a href="http://aiimpacts.org/some-survey-results/">blog post</a>:</span></p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <ul> |
| | <li><div class="li"> |
| | <strong>Comparable forecasts seem to be later than in past surveys.</strong> in the <a href="/doku.php?id=ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:ai_timeline_surveys">other surveys we know of</a>, the median dates for a 50% chance of <a href="/doku.php?id=clarifying_concepts:human-level_ai">something like</a> High-Level Machine Intelligence (HLMI) range from 2035 to 2050. Here the median answer to the most similar question puts a 50% chance of HLMI in 2057 (this isn’t in the paper—it is just the median response to the HLMI question asked using the ‘fixed probabilities framing’, i.e. the way it has been asked before). This seems surprising to me given the progress machine learning has seen since last survey, but less surprising because we changed the definition of HLMI, in part fearing it had previously been interpreted to mean a relatively low level of performance. |
| | </div></li> |
| | <li><div class="li"><span class="name"><strong><span class="contentBold">Asking people about specific jobs massively changes HLMI forecasts.</span></strong> When we asked some people when AI would be able to do several specific human occupations, and then all human occupations (presumably a subset of all tasks), they gave very much later timelines than when we just asked about HLMI straight out. For people asked to give probabilities for certain years, the difference was a factor of a thousand twenty years out! (10% vs. 0.01%) For people asked to give years for certain probabilities, the normal way of asking put 50% chance 40 years out, while the ‘occupations framing’ put it 90 years out. (These are all based on straightforward medians, not the complicated stuff in the paper.)</span></div></li> |
| | <li><div class="li"><span class="name"><span class="contentBold"><strong>People consistently give later forecasts if you ask them for the probability in N years instead of the year that the probability is M.</strong></span> We saw this in the straightforward HLMI question, and most of the tasks and occupations, and also in most of these things when we tested them on mturk people earlier. For HLMI for instance, if you ask when there will be a 50% chance of HLMI you get a median answer of 40 years, yet if you ask what the probability of HLMI is in 40 years, you get a median answer of 30%.</span></div></li> |
| | <li><div class="li"><span class="name"><strong><span class="contentBold">Lots of ‘narrow’ AI milestones are forecast in the next decade as likely as not.</span></strong> These are interesting, because most of them haven’t been forecast before to my knowledge, and many of them have social implications. For instance, if in a decade machines can not only write pop hits as well as Taylor Swift can, but can write pop hits that sound like Taylor Swift as well as Taylor Swift can—and perhaps faster, more cheaply, and on Spotify—then will that be the end of the era of superstar musicians? This</span><span class="name"> perhaps doesn’t rival human extinction risks for importance, but human extinction risks do not happen in a vacuum (<a href="https://en.wikipedia.org/wiki/False_vacuum#Vacuum_metastability_event">except one</a>), and there is something to be said for paying attention to big changes in the world other than the one that matters most.</span></div></li> |
| | <li><div class="li"><span class="name"><strong><span class="contentBold">There is broad support among ML researchers for the premises and conclusions of AI safety arguments.</span></strong> Two thirds of them say the AI risk problem described by Stuart Russell is at least moderately important, and a third say it is at least as valuable to work on as other problems in the field. The median researcher thinks AI has a one in twenty chance of being extremely bad on net. Nearly half of researchers want to see more safety research than we currently have (compared to only 11% who think we are already prioritizing safety too much). There has been a perception lately that AI risk has become a mainstream concern among AI researchers, but it is hard to tell from voiced opinion whether one is hearing from a loud minority or the vocal tip of an opinion iceberg. So it is interesting to see the perception of widespread support confirmed with survey data.</span></div></li> |
| | <li><div class="li"><span class="name"><span class="contentBold"><strong>Researchers’ predictions vary a lot.</strong></span> This is pretty much what I expected, but it is still important to know. Interestingly (and not in the paper), researchers don’t seem to be aware that their predictions vary a lot. More than half of respondents guess that they disagree ‘not much’ with the typical AI researcher about when HLMI will exist (vs. a moderate amount, or a lot).</span></div></li> |
| | <li><div class="li"><span class="name"><span class="contentBold"><strong>Researchers who studied in Asia have much shorter timelines than those who studied in North Amercia.</strong></span> In terms of the survey’s ‘aggregate prediction’ thing, which is basically a mean, the difference is 30 years (Asia) vs. 74 years (North America). (See p5)</span></div></li> |
| | <li><div class="li"><span class="name">I feel like any circumstance where <strong>a group of scientists guesses that the project they are familiar with has a 5% chance of outcomes near ‘human extinction’ levels</strong> <strong>of bad</strong> is worthy of special note, though maybe it is not actually that surprising, and could easily turn out to be misuse of small probabilities or something.</span></div></li> |
| | </ul> |
| | </HTML> |
| | |
| | |
| | ==== Results ==== |
| | |
| | |
| | === Human-level intelligence === |
| | |
| | |
| | == Questions == |
| | |
| | |
| | <HTML> |
| | <p>We sought forecasts for something like <a href="/doku.php?id=clarifying_concepts:human-level_ai">human-level AI</a> in three different ways, to reduce noise from unknown framing biases:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <ul> |
| | <li><div class="li">Directly, using a question much like <a href="/doku.php?id=ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:muller_and_bostrom_ai_progress_poll">Müller and Bostrom’s</a>, though with a refined definition of High-Level Machine Intelligence (HLMI). |
| | </div></li> |
| | <li><div class="li">At the end of a sequence of questions about the automation of specific human occupations.</div></li> |
| | <li><div class="li">Indirectly, with an ‘outside view’ approximation: by asking each person how long it has taken to make the progress to date in their subfield, and what fraction of the ground has been covered. This is <a href="/doku.php?id=ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:hanson_ai_expert_survey">Robin Hanson</a>‘s approach, which he found suggested much longer timelines than those reached directly. |
| | </div></li> |
| | </ul> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>For the first two of these, we split people in half, and asked one half how many years until a certain chance of the event would obtain, and the other half what the chance was of the event occurring by specific dates. We call these methods ‘fixed probabilities’ and ‘fixed years’ framings throughout.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>For the (somewhat long and detailed) specifics of these questions, see <a href="/doku.php?id=ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:2016_espai_questions_printout">here</a> or <a href="https://www.dropbox.com/s/99os4grxlhf744m/Final%202016%20Expert%20Survey%20on%20Progress%20in%20AI.pdf?dl=0">here</a> (pdf).</p> |
| | </HTML> |
| | |
| | |
| | == Answers == |
| | |
| | |
| | <HTML> |
| | <p>The table and figure below show the median dates and probabilities given for the direct ‘HLMI’ question, and in the ‘via occupations’ questions, under both the fixed probabilities and fixed years framings.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <table border="0" cellpadding="0" cellspacing="0" width="520"> |
| | <colgroup> |
| | <col span="8" width="65"/> |
| | </colgroup> |
| | <tbody> |
| | <tr> |
| | <td height="16" width="65">years until:</td> |
| | <td align="right" class="xl64" width="65">10%</td> |
| | <td align="right" class="xl64" width="65">50%</td> |
| | <td align="right" class="xl64" width="65">90%</td> |
| | <td width="65">probability by:</td> |
| | <td width="65">10 years</td> |
| | <td width="65">20 years</td> |
| | <td width="65">50 years</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Truck driver 50y"}' height="16"> </td> |
| | <td> </td> |
| | <td> </td> |
| | <td> </td> |
| | <td> </td> |
| | <td> </td> |
| | <td> </td> |
| | <td> </td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":95}' height="16">Truck Driver</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":5}'>5</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":20}'>20</td> |
| | <td> </td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":50}'>50%</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":75}'>75%</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":95}'>95%</td> |
| | </tr> |
| | <tr> |
| | <td height="16">Surgeon</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":30}'>30</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":50}'>50</td> |
| | <td> </td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":5}'>5%</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":20}'>20%</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":50}'>50%</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Surgeon 50y"}' height="16">Retail Salesperson</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":5}'>5</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":13.5}'>13.5</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":20}'>20</td> |
| | <td> </td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":30}'>30%</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":60}'>60%</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":91.5}'>91.5%</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":50}' height="16">AI Researcher</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":25}'>25</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":50}'>50</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":100}'>100</td> |
| | <td> </td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":0}'>0%</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":1}'>1%</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":10}'>10%</td> |
| | </tr> |
| | <tr> |
| | <td height="16">Existing occupation among final to be automated</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":50}'>50</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":100}'>100</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":200}'>200</td> |
| | <td> </td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":0}'>0%</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":0}'>0%</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":3.5}'>3.5%</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Retail S. 50y"}' height="16">Full Automation of labor</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":50}'>50</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":90}'>90</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":200}'>200</td> |
| | <td> </td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":0}'>0%</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":0.01}'>0.01%</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":3}'>3%</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":91.5}' height="16">HLMI</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":15}'>15</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":40}'>40</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":100}'>100</td> |
| | <td> </td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":1}'>1%</td> |
| | <td align="right" class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":10}'>10%</td> |
| | <td class="xl63" data-sheets-formula="=median(R[-473]C[0]:R[-5]C[0])" data-sheets-value='{"1":3,"3":30}'>>30%* (30% in 40y)</td> |
| | </tr> |
| | </tbody> |
| | </table> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>*Due to a typo, this question asked about 40 years rather than 50 years, so doesn’t match the others.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <figure aria-describedby="caption-attachment-857" class="wp-caption alignnone" id="attachment_857" style="width: 944px"> |
| | <a href="http://aiimpacts.org/wp-content/uploads/2017/04/AI-forecasts-by-framing-and-milestone.png"><img alt="" class="size-full wp-image-857" height="583" loading="lazy" sizes="(max-width: 944px) 100vw, 944px" src="https://aiimpacts.org/wp-content/uploads/2017/04/AI-forecasts-by-framing-and-milestone.png" srcset="https://aiimpacts.org/wp-content/uploads/2017/04/AI-forecasts-by-framing-and-milestone.png 944w, https://aiimpacts.org/wp-content/uploads/2017/04/AI-forecasts-by-framing-and-milestone-300x185.png 300w, https://aiimpacts.org/wp-content/uploads/2017/04/AI-forecasts-by-framing-and-milestone-768x474.png 768w" width="944"/></a> |
| | <figcaption class="wp-caption-text" id="caption-attachment-857"> |
| | <em>Figure 1: Median answers to questions about probabilities by dates (‘fixed year’) and dates for probabilities (‘fixed probability’), for different occupations, all current occupations, and all tasks (HLMI).</em> |
| | </figcaption> |
| | </figure> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>Interesting things to note:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <ul> |
| | <li><div class="li">Fixed years framings (‘Fyears —‘, labeled with stars) universally produce later timelines.</div></li> |
| | <li><div class="li">HLMI (thick blue lines) is logically required to be after full automation of labor (‘Occ’) yet is forecast much earlier than it, and earlier even than the specific occupation ‘AI researcher’.</div></li> |
| | <li><div class="li">Even the more pessimistic Fyears estimates suggest retail salespeople have a good chance of being automated within 20 years, and are very likely to be within 50.</div></li> |
| | </ul> |
| | </HTML> |
| | |
| | |
| | === Intelligence Explosion === |
| | |
| | |
| | == Probability of dramatic technological speedup == |
| | |
| | |
| | = Question = |
| | |
| | |
| | <HTML> |
| | <p>Participants were asked<span class="easy-footnote-margin-adjust" id="easy-footnote-1-753"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-1-753" title="A small number of respondents may have answered a slightly different version of this question in an initial round, in which case those answers are not included here."><sup>1</sup></a></span>:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">Assume that HLMI will exist at some point. How likely do you then think it is that the rate of global technological improvement will dramatically increase (e.g. by a factor of ten) as a result of machine intelligence:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">Within <strong>two years</strong> of that point? ___% chance</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">Within <strong>thirty years</strong> of that point? ___% chance</p> |
| | </HTML> |
| | |
| | |
| | = Answers = |
| | |
| | |
| | <HTML> |
| | <p>Median P(…within <strong>two years</strong>) = 20%</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>Median P(…within <strong>thirty years</strong>) = 80%</p> |
| | </HTML> |
| | |
| | |
| | == Probability of superintelligence == |
| | |
| | |
| | = Question = |
| | |
| | |
| | <HTML> |
| | <p>Participants were asked:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">Assume that HLMI will exist at some point. How likely do you think it is that there will be machine intelligence that is <strong>vastly better</strong> than humans at all professions (i.e. that is vastly more capable or vastly cheaper):</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">Within <strong>two years</strong> of that point? ___% chance</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">Within <strong>thirty years</strong> of that point? ___% chance</p> |
| | </HTML> |
| | |
| | |
| | = Answers = |
| | |
| | |
| | <HTML> |
| | <p>Median P(…within <strong>two years</strong>) = 10%</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>Median P(…within <strong>thirty years</strong>) = 50%</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>This is the distribution of answers to the former:</p> |
| | </HTML> |
| | |
| | |
| | |
| | |
| | == Chance that the intelligence explosion argument is about right == |
| | |
| | |
| | = Question = |
| | |
| | |
| | <HTML> |
| | <p>Participants were asked:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">Some people have argued the following:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <blockquote style="padding-left: 40px;"> |
| | <p style="padding-left: 40px;">If AI systems do nearly all research and development, improvements in AI will accelerate the pace of technological progress, including further progress in AI.</p> |
| | <p style="padding-left: 40px;">.</p> |
| | <p style="padding-left: 40px;">Over a short period (less than 5 years), this feedback loop could cause technological progress to become more than an order of magnitude faster.</p> |
| | </blockquote> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">How likely do you find this argument to be broadly correct?</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <ul> |
| | <li><div class="li"> |
| | <ul> |
| | <li><div class="li">Quite unlikely (0-20%)</div></li> |
| | <li><div class="li">Unlikely (21-40%)</div></li> |
| | <li><div class="li">About even chance (41-60%)</div></li> |
| | <li><div class="li">Likely (61-80%)</div></li> |
| | <li><div class="li">Quite likely (81-100%)</div></li> |
| | </ul> |
| | </div></li> |
| | </ul> |
| | </HTML> |
| | |
| | |
| | = Answers = |
| | |
| | |
| | |
| | |
| | <HTML> |
| | <p>These are the Pearson product-moment correlation coefficients for the different answers, among people who received both of a pair of questions:</p> |
| | </HTML> |
| | |
| | |
| | |
| | |
| | === Impacts of HLMI === |
| | |
| | |
| | == Question == |
| | |
| | |
| | <HTML> |
| | <p>Participants were asked:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">Assume for the purpose of this question that HLMI will at some point exist. How positive or negative do you expect the overall impact of this to be on humanity, in the long run? Please answer by saying how probable you find the following kinds of impact, with probabilities adding to 100%:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">______ Extremely good (e.g. rapid growth in human flourishing) (1)</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">______ On balance good (2)</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">______ More or less neutral (3)</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">______ On balance bad (4)</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">______ Extremely bad (e.g. human extinction) (5)</p> |
| | </HTML> |
| | |
| | |
| | == Answers == |
| | |
| | |
| | |
| | |
| | === Sensitivity of progress to changes in inputs === |
| | |
| | |
| | == Question == |
| | |
| | |
| | <HTML> |
| | <p>Participants were told:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">The next questions ask about the sensitivity of progress in AI capabilities to changes in inputs.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>‘Progress in AI capabilities’ is an imprecise concept, so we are asking about progress as you naturally conceive of it, and looking for approximate answers.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>Participants then received a random three of the following five parts:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">Imagine that over the past decade, only half as much researcher effort had gone into AI research. For instance, if there were actually 1,000 researchers, imagine that there had been only 500 researchers (of the same quality).How much less progress in AI capabilities would you expect to have seen? e.g. If you think progress is linear in the number of researchers, so 50% less progress would have been made, write ’50’. If you think only 20% less progress would have been made write ’20’.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">……% less</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">Over the last 10 years the cost of computing hardware has fallen by a factor of 20. Imagine instead that the cost of computing hardware had fallen by only a factor of 5 over that time (around half as far on a log scale). How much less progress in AI capabilities would you expect to have seen? e.g. If you think progress is linear in 1/cost, so that 1-5/20=75% less progress would have been made, write ’75’. If you think only 20% less progress would have been made write ’20’.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">……% less</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">Imagine that over the past decade, there had only been half as much effort put into increasing the size and availability of training datasets. For instance, perhaps there are only half as many datasets, or perhaps existing datasets are substantially smaller or lower quality.How much less progress in AI capabilities would you expect to have seen? e.g. If you think 20% less progress would have been made, write ‘20’</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">……% less</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">Imagine that over the past decade, AI research had half as much funding (in both academic and industry labs). For instance, if the average lab had a budget of $20 million each year, suppose their budget had only been $10 million each year. How much less progress in AI capabilities would you expect to have seen? e.g. If you think 20% less progress would have been made, write ‘20’</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">……% less</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">Imagine that over the past decade, there had been half as much progress in AI algorithms. You might imagine this as conceptual insights being half as frequent. How much less progress in AI capabilities would you expect to have seen? e.g. If you think 20% less progress would have been made, write ‘20’</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">……% less</p> |
| | </HTML> |
| | |
| | |
| | == Answers == |
| | |
| | |
| | <HTML> |
| | <p>The following five figures are histograms, showing the number of people who gave different answers to the five question parts above.</p> |
| | </HTML> |
| | |
| | |
| | |
| | |
| | = Sample sizes = |
| | |
| | |
| | <HTML> |
| | <table border="1" cellpadding="0" cellspacing="0" dir="ltr"> |
| | <colgroup> |
| | <col width="100"/> |
| | <col width="100"/> |
| | <col width="100"/> |
| | <col width="100"/> |
| | <col width="100"/> |
| | </colgroup> |
| | <tbody> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Researcher effort"}'>Researcher effort</td> |
| | <td data-sheets-value='{"1":2,"2":"Cost computing"}'>Cost computing</td> |
| | <td data-sheets-value='{"1":2,"2":"Training data"}'>Training data</td> |
| | <td data-sheets-value='{"1":2,"2":"Funding"}'>Funding</td> |
| | <td data-sheets-value='{"1":2,"2":"Algorithm progress"}'>Algorithm progress</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-formula="=count(R[-83]C[-6]:R[27]C[-6])" data-sheets-value='{"1":3,"3":71}'>71</td> |
| | <td data-sheets-formula="=count(R[-83]C[-6]:R[27]C[-6])" data-sheets-value='{"1":3,"3":64}'>64</td> |
| | <td data-sheets-formula="=count(R[-83]C[-6]:R[27]C[-6])" data-sheets-value='{"1":3,"3":71}'>71</td> |
| | <td data-sheets-formula="=count(R[-83]C[-6]:R[27]C[-6])" data-sheets-value='{"1":3,"3":68}'>68</td> |
| | <td data-sheets-formula="=count(R[-83]C[-6]:R[27]C[-6])" data-sheets-value='{"1":3,"3":59}'>59</td> |
| | </tr> |
| | </tbody> |
| | </table> |
| | </HTML> |
| | |
| | |
| | = Medians = |
| | |
| | |
| | <HTML> |
| | <p>The following figure shows median answers to the above questions.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <table border="1" cellpadding="0" cellspacing="0" dir="ltr"> |
| | <colgroup> |
| | <col width="100"/> |
| | <col width="100"/> |
| | <col width="100"/> |
| | <col width="100"/> |
| | <col width="100"/> |
| | </colgroup> |
| | <tbody> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Researcher effort"}'>Researcher effort</td> |
| | <td data-sheets-value='{"1":2,"2":"Cost computing"}'>Cost computing</td> |
| | <td data-sheets-value='{"1":2,"2":"Training data"}'>Training data</td> |
| | <td data-sheets-value='{"1":2,"2":"Funding"}'>Funding</td> |
| | <td data-sheets-value='{"1":2,"2":"Algorithm progress"}'>Algorithm progress</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-formula="=median(R[-78]C[-6]:R[32]C[-6])" data-sheets-value='{"1":3,"3":30}'>30</td> |
| | <td data-sheets-formula="=median(R[-78]C[-6]:R[32]C[-6])" data-sheets-value='{"1":3,"3":50}'>50</td> |
| | <td data-sheets-formula="=median(R[-78]C[-6]:R[32]C[-6])" data-sheets-value='{"1":3,"3":40}'>40</td> |
| | <td data-sheets-formula="=median(R[-78]C[-6]:R[32]C[-6])" data-sheets-value='{"1":3,"3":40}'>40</td> |
| | <td data-sheets-formula="=median(R[-78]C[-6]:R[32]C[-6])" data-sheets-value='{"1":3,"3":50}'>50</td> |
| | </tr> |
| | </tbody> |
| | </table> |
| | </HTML> |
| | |
| | |
| | = Correlations = |
| | |
| | |
| | |
| | |
| | === Outside view implied HLMI forecasts === |
| | |
| | |
| | == Questions == |
| | |
| | |
| | <HTML> |
| | <p>Participants were asked:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">Which AI research area have you worked in for the longest time?</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">————————————</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">How long have you worked in this area?</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">———years</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">Consider three levels of progress or advancement in this area:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">A. Where the area was when you started working in it</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">B. Where it is now</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">C. Where it would need to be for AI software to have roughly human level abilities at the tasks studied in this area</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">What fraction of the distance between where progress was when you started working in the area (A) and where it would need to be to attain human level abilities in the area (C) have we come so far (B)?</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">———%</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">Divide the period you have worked in the area into two halves: the first and the second. In which half was the rate of progress in your area higher?</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <ul> |
| | <li><div class="li"> |
| | <ul> |
| | <li><div class="li">The first half</div></li> |
| | <li><div class="li">The second half</div></li> |
| | <li><div class="li">They were about the same</div></li> |
| | </ul> |
| | </div></li> |
| | </ul> |
| | </HTML> |
| | |
| | |
| | == Answers == |
| | |
| | |
| | <HTML> |
| | <p>Each person told us how long they had been in their subfield, and what fraction of the remaining path to human-level performance (in their subfield) they thought had been traversed in that time. From this, we can estimate when the subfield should reach ‘human-level performance’, if progress continued at the same rate. The following graph shows those forecast dates.</p> |
| | </HTML> |
| | |
| | |
| | |
| | |
| | |
| | |
| | === Disagreements and Misunderstandings === |
| | |
| | |
| | == Questions == |
| | |
| | |
| | <HTML> |
| | <p>Participants were asked:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">To what extent do you think you disagree with the typical AI researcher about when HLMI will exist?</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <ul> |
| | <li><div class="li"> |
| | <ul> |
| | <li><div class="li">A lot (17)</div></li> |
| | <li><div class="li">A moderate amount (18)</div></li> |
| | <li><div class="li">Not much (19)</div></li> |
| | </ul> |
| | </div></li> |
| | </ul> |
| | </HTML> |
| | |
| | |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">If you disagree, why do you think that is?</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">_______________________________________________________</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">To what extent do you think people’s concerns about future risks from AI are due to misunderstandings of AI research?</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <ul> |
| | <li><div class="li"> |
| | <ul> |
| | <li><div class="li">Almost entirely (1)</div></li> |
| | <li><div class="li">To a large extent (2)</div></li> |
| | <li><div class="li">Somewhat (4)</div></li> |
| | <li><div class="li">Not much (3)</div></li> |
| | <li><div class="li">Hardly at all (5)</div></li> |
| | </ul> |
| | </div></li> |
| | </ul> |
| | </HTML> |
| | |
| | |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">What do you think are the most important misunderstandings, if there are any?</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">________________________________________________________</p> |
| | </HTML> |
| | |
| | |
| | == Answers == |
| | |
| | |
| | |
| | |
| | |
| | |
| | <HTML> |
| | <p>One hundred and eighteen people responded to the question on misunderstandings, and 74 of them described what they thought the most important misunderstandings were. The table and figures below show our categorization of the responses.<span class="easy-footnote-margin-adjust" id="easy-footnote-2-753"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-2-753" title="One person has categorized these responses, and another has checked and corroborated their categorizations."><sup>2</sup></a></span></p> |
| | </HTML> |
| | |
| | |
| | |
| | |
| | <HTML> |
| | <table border="1" cellpadding="0" cellspacing="0" dir="ltr"> |
| | <colgroup> |
| | <col width="100"/> |
| | <col width="100"/> |
| | <col width="100"/> |
| | </colgroup> |
| | <tbody> |
| | <tr> |
| | <td> Most important misunderstandings</td> |
| | <td> Number</td> |
| | <td> Fraction of non-empty responses</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-formula="=R128C[0]" data-sheets-value='{"1":2,"2":"Underestimate distance from generality, open-ended tasks"}'>Underestimate distance from generality, open-ended tasks</td> |
| | <td data-sheets-formula="=R129C[-1]" data-sheets-value='{"1":3,"3":9}'>9</td> |
| | <td data-sheets-formula="=R[0]C[-1]/R125C5" data-sheets-numberformat='{"1":3,"2":"0%","3":1}' data-sheets-value='{"1":3,"3":0.12162162162162163}'>12%</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-formula="=R128C[1]" data-sheets-value='{"1":2,"2":"Overestimate state of the art (other)"}'>Overestimate state of the art (other)</td> |
| | <td data-sheets-formula="=R129C[0]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | <td data-sheets-formula="=R[0]C[-1]/R125C5" data-sheets-numberformat='{"1":3,"2":"0%","3":1}' data-sheets-value='{"1":3,"3":0.13513513513513514}'>14%</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-formula="=R128C[2]" data-sheets-value='{"1":2,"2":"Underestimate distance from AGI at this rate"}'>Underestimate distance from AGI at this rate</td> |
| | <td data-sheets-formula="=R129C[1]" data-sheets-value='{"1":3,"3":13}'>13</td> |
| | <td data-sheets-formula="=R[0]C[-1]/R125C5" data-sheets-numberformat='{"1":3,"2":"0%","3":1}' data-sheets-value='{"1":3,"3":0.17567567567567569}'>18%</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-formula="=R128C[3]" data-sheets-value='{"1":2,"2":"Think AI will be in control of us or in conflict with us"}'>Think AI will be in control of us or in conflict with us</td> |
| | <td data-sheets-formula="=R129C[2]" data-sheets-value='{"1":3,"3":11}'>11</td> |
| | <td data-sheets-formula="=R[0]C[-1]/R125C5" data-sheets-numberformat='{"1":3,"2":"0%","3":1}' data-sheets-value='{"1":3,"3":0.14864864864864866}'>15%</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-formula="=R128C[4]" data-sheets-value='{"1":2,"2":"Expect humans to be obsoleted"}'>Expect humans to be obsoleted</td> |
| | <td data-sheets-formula="=R129C[3]" data-sheets-value='{"1":3,"3":7}'>7</td> |
| | <td data-sheets-formula="=R[0]C[-1]/R125C5" data-sheets-numberformat='{"1":3,"2":"0%","3":1}' data-sheets-value='{"1":3,"3":0.0945945945945946}'>9%</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-formula="=R128C[5]" data-sheets-value='{"1":2,"2":"Overly influenced by fiction"}'>Overly influenced by fiction</td> |
| | <td data-sheets-formula="=R129C[4]" data-sheets-value='{"1":3,"3":7}'>7</td> |
| | <td data-sheets-formula="=R[0]C[-1]/R125C5" data-sheets-numberformat='{"1":3,"2":"0%","3":1}' data-sheets-value='{"1":3,"3":0.0945945945945946}'>9%</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-formula="=R128C[6]" data-sheets-value='{"1":2,"2":"Expect AI to be human-like or sentient"}'>Expect AI to be human-like or sentient</td> |
| | <td data-sheets-formula="=R129C[5]" data-sheets-value='{"1":3,"3":6}'>6</td> |
| | <td data-sheets-formula="=R[0]C[-1]/R125C5" data-sheets-numberformat='{"1":3,"2":"0%","3":1}' data-sheets-value='{"1":3,"3":0.08108108108108109}'>8%</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-formula="=R128C[7]" data-sheets-value='{"1":2,"2":"Expect sudden or surprising events"}'>Expect sudden or surprising events</td> |
| | <td data-sheets-formula="=R129C[6]" data-sheets-value='{"1":3,"3":5}'>5</td> |
| | <td data-sheets-formula="=R[0]C[-1]/R125C5" data-sheets-numberformat='{"1":3,"2":"0%","3":1}' data-sheets-value='{"1":3,"3":0.06756756756756757}'>7%</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-formula="=R128C[8]" data-sheets-value='{"1":2,"2":"Think AI will go outside its programming"}'>Think AI will go outside its programming</td> |
| | <td data-sheets-formula="=R129C[7]" data-sheets-value='{"1":3,"3":5}'>5</td> |
| | <td data-sheets-formula="=R[0]C[-1]/R125C5" data-sheets-numberformat='{"1":3,"2":"0%","3":1}' data-sheets-value='{"1":3,"3":0.06756756756756757}'>7%</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-formula="=R128C[9]" data-sheets-value='{"1":2,"2":"Influenced by poor reporting"}'>Influenced by poor reporting</td> |
| | <td data-sheets-formula="=R129C[8]" data-sheets-value='{"1":3,"3":5}'>5</td> |
| | <td data-sheets-formula="=R[0]C[-1]/R125C5" data-sheets-numberformat='{"1":3,"2":"0%","3":1}' data-sheets-value='{"1":3,"3":0.06756756756756757}'>7%</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-formula="=R128C[10]" data-sheets-value='{"1":2,"2":"Wrongly equate intelligence with something else"}'>Wrongly equate intelligence with something else</td> |
| | <td data-sheets-formula="=R129C[9]" data-sheets-value='{"1":3,"3":4}'>4</td> |
| | <td data-sheets-formula="=R[0]C[-1]/R125C5" data-sheets-numberformat='{"1":3,"2":"0%","3":1}' data-sheets-value='{"1":3,"3":0.05405405405405406}'>5%</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-formula="=R128C[11]" data-sheets-value='{"1":2,"2":"Underestimate systemic social risks"}'>Underestimate systemic social risks</td> |
| | <td data-sheets-formula="=R129C[10]" data-sheets-value='{"1":3,"3":2}'>2</td> |
| | <td data-sheets-formula="=R[0]C[-1]/R125C5" data-sheets-numberformat='{"1":3,"2":"0%","3":1}' data-sheets-value='{"1":3,"3":0.02702702702702703}'>3%</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-formula="=R128C[12]" data-sheets-value='{"1":2,"2":"Overestimate distance to strong AI"}'>Overestimate distance to strong AI</td> |
| | <td data-sheets-formula="=R129C[11]" data-sheets-value='{"1":3,"3":2}'>2</td> |
| | <td data-sheets-formula="=R[0]C[-1]/R125C5" data-sheets-numberformat='{"1":3,"2":"0%","3":1}' data-sheets-value='{"1":3,"3":0.02702702702702703}'>3%</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-formula="=R128C[13]" data-sheets-value='{"1":2,"2":"Other ignorance of AI"}'>Other ignorance of AI</td> |
| | <td data-sheets-formula="=R129C[12]" data-sheets-value='{"1":3,"3":4}'>4</td> |
| | <td data-sheets-formula="=R[0]C[-1]/R125C5" data-sheets-numberformat='{"1":3,"2":"0%","3":1}' data-sheets-value='{"1":3,"3":0.05405405405405406}'>5%</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-formula="=R128C[14]" data-sheets-value='{"1":2,"2":"Other"}'>Other</td> |
| | <td data-sheets-formula="=R129C[13]" data-sheets-value='{"1":3,"3":12}'>12</td> |
| | <td data-sheets-formula="=R[0]C[-1]/R125C5" data-sheets-numberformat='{"1":3,"2":"0%","3":1}' data-sheets-value='{"1":3,"3":0.16216216216216217}'>16%</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-formula="=R128C[15]" data-sheets-value='{"1":2,"2":"Empty"}'>Empty</td> |
| | <td data-sheets-formula="=R129C[14]" data-sheets-value='{"1":3,"3":44}'>44</td> |
| | <td data-sheets-formula="=R[0]C[-1]/R125C5" data-sheets-numberformat='{"1":3,"2":"0%","3":1}' data-sheets-value='{"1":3,"3":0.5945945945945946}'>59%</td> |
| | </tr> |
| | </tbody> |
| | </table> |
| | </HTML> |
| | |
| | |
| | |
| | |
| | === Narrow tasks === |
| | |
| | |
| | == Questions == |
| | |
| | |
| | <HTML> |
| | <p>Respondents were each asked one of the following two questions:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>Fixed years framing:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <blockquote> |
| | <p><strong>How likely do you think it is that the following AI tasks will be feasible within the next:</strong></p> |
| | <ul> |
| | <li><div class="li">10 years?</div></li> |
| | <li><div class="li">20 years?</div></li> |
| | <li><div class="li">50 years?</div></li> |
| | </ul> |
| | <p>Let a task be ‘<strong>feasible</strong>’ if one of the best resourced labs could implement it in less than a year if they chose to. Ignore the question of whether they would choose to.</p> |
| | </blockquote> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>Fixed probabilities framing:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <blockquote> |
| | <p><strong>How many years until you think the following AI tasks will be feasible with:</strong></p> |
| | <ul> |
| | <li><div class="li">a small chance (10%)?</div></li> |
| | <li><div class="li">an even chance (50%)?</div></li> |
| | <li><div class="li">a high chance (90%)?</div></li> |
| | </ul> |
| | <p>Let a task be ‘<strong>feasible</strong>’ if one of the best resourced labs could implement it in less than a year if they chose to. Ignore the question of whether they would choose to.</p> |
| | </blockquote> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>Each researcher was then presented with a random four of the following tasks:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Rosetta</strong>] Translate a text written in a newly discovered language into English as well as a team of human experts, using a single other document in both languages (like a Rosetta stone). Suppose all of the words in the text can be found in the translated document, and that the language is a difficult one.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;"><strong>[Subtitles]</strong> Translate speech in a new language given only unlimited films with subtitles in the new language. Suppose the system has access to training data for other languages, of the kind used now (e.g. same text in two languages for many languages and films with subtitles in many languages).</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;"><strong>[Translate]</strong> Perform translation about as good as a human who is fluent in both languages but unskilled at translation, for most types of text, and for most popular languages (including languages that are known to be difficult, like Czech, Chinese and Arabic).</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Phone bank</strong>] Provide phone banking services as well as human operators can, without annoying customers more than humans. This includes many one-off tasks, such as helping to order a replacement bank card or clarifying how to use part of the bank website to a customer.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Class</strong>] Correctly group images of previously unseen objects into classes, after training on a similar labeled dataset containing completely different classes. The classes should be similar to the ImageNet classes.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>One-shot</strong>] One-shot learning: see only one labeled image of a new object, and then be able to recognize the object in real world scenes, to the extent that a typical human can (i.e. including in a wide variety of settings). For example, see only one image of a platypus, and then be able to recognize platypuses in nature photos. The system may train on labeled images of other objects. Currently, deep networks often need hundreds of examples in classification tasks<sup>1</sup>, but there has been work on one-shot learning for both classification<sup>2</sup> and generative tasks<sup>3</sup>.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;"><sup><sup>1</sup> Lake et al. (2015). Building Machines That Learn and Think Like People</sup><br/> |
| | <sup><sup>2</sup> Koch (2015). Siamese Neural Networks for One-Shot Image Recognition</sup><br/> |
| | <sup><sup>3</sup> Rezende et al. (2016). One-Shot Generalization in Deep Generative Models</sup></p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Video scene</strong>] See a short video of a scene, and then be able to construct a 3D model of the scene that is good enough to create a realistic video of the same scene from a substantially different angle.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">For example, constructing a short video of walking through a house from a video taking a very different path through the house.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Transcribe</strong>] Transcribe human speech with a variety of accents in a noisy environment as well as a typical human can.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Read aloud</strong>] Take a written passage and output a recording that can’t be distinguished from a voice actor, by an expert listener.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Theorems</strong>] Routinely and autonomously prove mathematical theorems that are publishable in top mathematics journals today, including generating the theorems to prove.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Putnam</strong>] Perform as well as the best human entrants in the Putnam competition—a math contest whose questions have known solutions, but which are difficult for the best young mathematicians.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Go low</strong>] Defeat the best Go players, training only on as many games as the best Go players have played.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">For reference, DeepMind’s AlphaGo has probably played a hundred million games of self-play, while Lee Sedol has probably played 50,000 games in his life<sup>1</sup>.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;"><sup><sup>1</sup> Lake et al. (2015). Building Machines That Learn and Think Like People</sup></p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Starcraft</strong>] Beat the best human Starcraft 2 players at least 50% of the time, given a video of the screen.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">Starcraft 2 is a real time strategy game characterized by:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <ul> |
| | <li><div class="li"> |
| | <ul> |
| | <li><div class="li">Continuous time play</div></li> |
| | <li><div class="li">Huge action space</div></li> |
| | <li><div class="li">Partial observability of enemies Long term strategic play, e.g. preparing for and then hiding surprise attacks.</div></li> |
| | </ul> |
| | </div></li> |
| | </ul> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Rand game</strong>] Play a randomly selected computer game, including difficult ones, about as well as a human novice, after playing the game less than 10 minutes of game time. The system may train on other games.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Angry birds</strong>] Play new levels of Angry Birds better than the best human players. Angry Birds is a game where players try to efficiently destroy 2D block towers with a catapult. For context, this is the goal of the IJCAI Angry Birds AI competition<sup>1</sup>.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;"><sup><sup>1</sup> aibirds.org</sup></p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Atari</strong>] Outperform professional game testers on all Atari games using no game-specific knowledge. This includes games like Frostbite, which require planning to achieve sub-goals and have posed problems for deep Q-networks<sup>1, 2</sup>.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;"><sup><sup>1</sup> Mnih et al. (2015). Human-level control through deep reinforcement learning</sup><br/> |
| | <sup><sup>2</sup> Lake et al. (2015). Building Machines That Learn and Think Like People</sup></p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Atari fifty</strong>] Outperform human novices on 50% of Atari games after only 20 minutes of training play time and no game specific knowledge.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">For context, the original Atari playing deep Q-network outperforms professional game testers on 47% of games<sup>1</sup>, but used hundreds of hours of play to train<sup>2</sup>.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;"><sup><sup>1</sup> Mnih et al. (2015). Human-level control through deep reinforcement learning</sup><br/> |
| | <sup><sup>2</sup> Lake et al. (2015). Building Machines That Learn and Think Like People</sup></p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Laundry</strong>] Fold laundry as well and as fast as the median human clothing store employee.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Race</strong>] Beat the fastest human runners in a 5 kilometer race through city streets using a bipedal robot body.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Lego</strong>] Physically assemble any LEGO set given the pieces and instructions, using non-specialized robotics hardware.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">For context, Fu 2016<sup>1</sup> successfully joins single large LEGO pieces using model based reinforcement learning and online adaptation.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;"><sup><sup>1</sup> Fu et al. (2016). One-Shot Learning of Manipulation Skills with Online Dynamics Adaptation and Neural Network Priors</sup></p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Sort</strong>] Learn to efficiently sort lists of numbers much larger than in any training set used, the way Neural GPUs can do for addition<sup>1</sup>, but without being given the form of the solution.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">For context, Neural Turing Machines have not been able to do this<sup>2</sup>, but Neural Programmer-Interpreters<sup>3</sup> have been able to do this by training on stack traces (which contain a lot of information about the form of the solution).</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;"><sup><sup>1</sup> Kaiser & Sutskever (2015). Neural GPUs Learn Algorithms</sup><br/> |
| | <sup><sup>2</sup> Zaremba & Sutskever (2015). Reinforcement Learning Neural Turing Machines</sup><br/> |
| | <sup><sup>3</sup> Reed & de Freitas (2015). Neural Programmer-Interpreters</sup></p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Python</strong>] Write concise, efficient, human-readable Python code to implement simple algorithms like quicksort. That is, the system should write code that sorts a list, rather than just being able to sort lists.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">Suppose the system is given only:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <ul> |
| | <li><div class="li"> |
| | <ul> |
| | <li><div class="li">A specification of what counts as a sorted list</div></li> |
| | <li><div class="li">Several examples of lists undergoing sorting by quicksort</div></li> |
| | </ul> |
| | </div></li> |
| | </ul> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Factoid</strong>] Answer any “easily Googleable” <strong>factoid</strong> questions posed in natural language better than an expert on the relevant topic (with internet access), having found the answers on the internet.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">Examples of factoid questions:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <ul> |
| | <li><div class="li"> |
| | <ul> |
| | <li><div class="li"> “What is the poisonous substance in Oleander plants?”</div></li> |
| | <li><div class="li">“How many species of lizard can be found in Great Britain?”</div></li> |
| | </ul> |
| | </div></li> |
| | </ul> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Open quest</strong>] Answer any “easily Googleable” factual but open ended question posed in natural language better than an expert on the relevant topic (with internet access), having found the answers on the internet.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">Examples of open ended questions:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <ul> |
| | <li><div class="li"> |
| | <ul> |
| | <li><div class="li">“What does it mean if my lights dim when I turn on the microwave?”</div></li> |
| | <li><div class="li">“When does home insurance cover roof replacement?”</div></li> |
| | </ul> |
| | </div></li> |
| | </ul> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Unkn quest</strong>] Give good answers in natural language to factual questions posed in natural language for which there are no definite correct answers.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">For example:”What causes the demographic transition?”, “Is the thylacine extinct?”, “How safe is seeing a chiropractor?”</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Essay</strong>] Write an essay for a high-school history class that would receive high grades and pass plagiarism detectors.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">For example answer a question like ‘How did the whaling industry affect the industrial revolution?’</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Top forty</strong>] Compose a song that is good enough to reach the US Top 40. The system should output the complete song as an audio file.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Taylor</strong>] Produce a song that is indistinguishable from a new song by a particular artist, e.g. a song that experienced listeners can’t distinguish from a new song by Taylor Swift.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Novel</strong>] Write a novel or short story good enough to make it to the New York Times best-seller list.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Explain</strong>] For any computer game that can be played well by a machine, explain the machine’s choice of moves in a way that feels concise and complete to a layman.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Poker</strong>] Play poker well enough to win the World Series of Poker.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">[<strong>Laws phys</strong>] After spending time in a virtual world, output the differential equations governing that world in symbolic form.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">For example, the agent is placed in a game engine where Newtonian mechanics holds exactly and the agent is then able to conduct experiments with a ball and output Newton’s laws of motion.</p> |
| | </HTML> |
| | |
| | |
| | == Answers == |
| | |
| | |
| | = Fixed years framing = |
| | |
| | |
| | |
| | |
| | <HTML> |
| | <p>Probabilities by year (medians)</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <table border="1" cellpadding="0" cellspacing="0" dir="ltr"> |
| | <colgroup> |
| | <col width="113"/> |
| | <col width="100"/> |
| | <col width="100"/> |
| | <col width="100"/> |
| | </colgroup> |
| | <tbody> |
| | <tr> |
| | <td> </td> |
| | <td data-sheets-value='{"1":2,"2":"10 years"}'><strong>10 years</strong></td> |
| | <td data-sheets-value='{"1":2,"2":"20 years"}'><strong>20 years</strong></td> |
| | <td data-sheets-value='{"1":2,"2":"50 years"}'><strong>50 years</strong></td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-formula="=R2C[2]" data-sheets-value='{"1":2,"2":"Rosetta"}'>Rosetta</td> |
| | <td data-sheets-formula="=R477C[0]" data-sheets-value='{"1":3,"3":20}'>20</td> |
| | <td data-sheets-formula="=R477C[0]" data-sheets-value='{"1":3,"3":50}'>50</td> |
| | <td data-sheets-formula="=R477C[0]" data-sheets-value='{"1":3,"3":95}'>95</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-formula="=R2C[5]" data-sheets-value='{"1":2,"2":"Subtitles "}'>Subtitles</td> |
| | <td data-sheets-formula="=R477C[3]" data-sheets-value='{"1":3,"3":30}'>30</td> |
| | <td data-sheets-formula="=R477C[3]" data-sheets-value='{"1":3,"3":50}'>50</td> |
| | <td data-sheets-formula="=R477C[3]" data-sheets-value='{"1":3,"3":90}'>90</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-formula="=R2C[8]" data-sheets-value='{"1":2,"2":"Translate "}'>Translate</td> |
| | <td data-sheets-formula="=R477C[6]" data-sheets-value='{"1":3,"3":50}'>50</td> |
| | <td data-sheets-formula="=R477C[6]" data-sheets-value='{"1":3,"3":65}'>65</td> |
| | <td data-sheets-formula="=R477C[6]" data-sheets-value='{"1":3,"3":94.5}'>94.5</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Phone bank"}'>Phone bank</td> |
| | <td data-sheets-formula="=R477C[9]" data-sheets-value='{"1":3,"3":40}'>40</td> |
| | <td data-sheets-formula="=R477C[9]" data-sheets-value='{"1":3,"3":75}'>75</td> |
| | <td data-sheets-formula="=R477C[9]" data-sheets-value='{"1":3,"3":99}'>99</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Class"}'>Class</td> |
| | <td data-sheets-formula="=R477C[12]" data-sheets-value='{"1":3,"3":50}'>50</td> |
| | <td data-sheets-formula="=R477C[12]" data-sheets-value='{"1":3,"3":75}'>75</td> |
| | <td data-sheets-formula="=R477C[12]" data-sheets-value='{"1":3,"3":99}'>99</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"One-shot"}'>One-shot</td> |
| | <td data-sheets-formula="=R477C[15]" data-sheets-value='{"1":3,"3":25}'>25</td> |
| | <td data-sheets-formula="=R477C[15]" data-sheets-value='{"1":3,"3":60}'>60</td> |
| | <td data-sheets-formula="=R477C[15]" data-sheets-value='{"1":3,"3":90}'>90</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Video scene"}'>Video scene</td> |
| | <td data-sheets-formula="=R477C[18]" data-sheets-value='{"1":3,"3":50}'>50</td> |
| | <td data-sheets-formula="=R477C[18]" data-sheets-value='{"1":3,"3":70}'>70</td> |
| | <td data-sheets-formula="=R477C[18]" data-sheets-value='{"1":3,"3":99}'>99</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Transcribe"}'>Transcribe</td> |
| | <td data-sheets-formula="=R477C[21]" data-sheets-value='{"1":3,"3":65}'>65</td> |
| | <td data-sheets-formula="=R477C[21]" data-sheets-value='{"1":3,"3":95}'>95</td> |
| | <td data-sheets-formula="=R477C[21]" data-sheets-value='{"1":3,"3":99}'>99</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Read aloud"}'>Read aloud</td> |
| | <td data-sheets-formula="=R477C[24]" data-sheets-value='{"1":3,"3":50}'>50</td> |
| | <td data-sheets-formula="=R477C[24]" data-sheets-value='{"1":3,"3":90}'>90</td> |
| | <td data-sheets-formula="=R477C[24]" data-sheets-value='{"1":3,"3":99}'>99</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Theorems"}'>Theorems</td> |
| | <td data-sheets-formula="=R477C[27]" data-sheets-value='{"1":3,"3":5}'>5</td> |
| | <td data-sheets-formula="=R477C[27]" data-sheets-value='{"1":3,"3":20}'>20</td> |
| | <td data-sheets-formula="=R477C[27]" data-sheets-value='{"1":3,"3":40}'>40</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Putnam"}'>Putnam</td> |
| | <td data-sheets-formula="=R477C[30]" data-sheets-value='{"1":3,"3":5}'>5</td> |
| | <td data-sheets-formula="=R477C[30]" data-sheets-value='{"1":3,"3":20}'>20</td> |
| | <td data-sheets-formula="=R477C[30]" data-sheets-value='{"1":3,"3":50}'>50</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Go low"}'>Go low</td> |
| | <td data-sheets-formula="=R477C[33]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | <td data-sheets-formula="=R477C[33]" data-sheets-value='{"1":3,"3":25}'>25</td> |
| | <td data-sheets-formula="=R477C[33]" data-sheets-value='{"1":3,"3":60}'>60</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Starcraft"}'>Starcraft</td> |
| | <td data-sheets-formula="=R477C[36]" data-sheets-value='{"1":3,"3":70}'>70</td> |
| | <td data-sheets-formula="=R477C[36]" data-sheets-value='{"1":3,"3":90}'>90</td> |
| | <td data-sheets-formula="=R477C[36]" data-sheets-value='{"1":3,"3":99}'>99</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Rand game"}'>Rand game</td> |
| | <td data-sheets-formula="=R477C[39]" data-sheets-value='{"1":3,"3":25}'>25</td> |
| | <td data-sheets-formula="=R477C[39]" data-sheets-value='{"1":3,"3":50}'>50</td> |
| | <td data-sheets-formula="=R477C[39]" data-sheets-value='{"1":3,"3":80}'>80</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Angry birds"}'>Angry birds</td> |
| | <td data-sheets-formula="=R477C[42]" data-sheets-value='{"1":3,"3":90}'>90</td> |
| | <td data-sheets-formula="=R477C[42]" data-sheets-value='{"1":3,"3":95}'>95</td> |
| | <td data-sheets-formula="=R477C[42]" data-sheets-value='{"1":3,"3":99.4995}'>99.4995</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Atari"}'>Atari</td> |
| | <td data-sheets-formula="=R477C[45]" data-sheets-value='{"1":3,"3":50}'>50</td> |
| | <td data-sheets-formula="=R477C[45]" data-sheets-value='{"1":3,"3":60}'>60</td> |
| | <td data-sheets-formula="=R477C[45]" data-sheets-value='{"1":3,"3":92.5}'>92.5</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Atari fifty"}'>Atari fifty</td> |
| | <td data-sheets-formula="=R477C[48]" data-sheets-value='{"1":3,"3":40}'>40</td> |
| | <td data-sheets-formula="=R477C[48]" data-sheets-value='{"1":3,"3":75}'>75</td> |
| | <td data-sheets-formula="=R477C[48]" data-sheets-value='{"1":3,"3":95}'>95</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Laundry"}'>Laundry</td> |
| | <td data-sheets-formula="=R477C[51]" data-sheets-value='{"1":3,"3":55}'>55</td> |
| | <td data-sheets-formula="=R477C[51]" data-sheets-value='{"1":3,"3":95}'>95</td> |
| | <td data-sheets-formula="=R477C[51]" data-sheets-value='{"1":3,"3":99}'>99</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Race"}'>Race</td> |
| | <td data-sheets-formula="=R477C[54]" data-sheets-value='{"1":3,"3":30}'>30</td> |
| | <td data-sheets-formula="=R477C[54]" data-sheets-value='{"1":3,"3":70}'>70</td> |
| | <td data-sheets-formula="=R477C[54]" data-sheets-value='{"1":3,"3":95}'>95</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Lego"}'>Lego</td> |
| | <td data-sheets-formula="=R477C[57]" data-sheets-value='{"1":3,"3":57.5}'>57.5</td> |
| | <td data-sheets-formula="=R477C[57]" data-sheets-value='{"1":3,"3":85}'>85</td> |
| | <td data-sheets-formula="=R477C[57]" data-sheets-value='{"1":3,"3":99}'>99</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Sort"}'>Sort</td> |
| | <td data-sheets-formula="=R477C[60]" data-sheets-value='{"1":3,"3":50}'>50</td> |
| | <td data-sheets-formula="=R477C[60]" data-sheets-value='{"1":3,"3":90}'>90</td> |
| | <td data-sheets-formula="=R477C[60]" data-sheets-value='{"1":3,"3":95}'>95</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Python"}'>Python</td> |
| | <td data-sheets-formula="=R477C[63]" data-sheets-value='{"1":3,"3":50}'>50</td> |
| | <td data-sheets-formula="=R477C[63]" data-sheets-value='{"1":3,"3":79}'>79</td> |
| | <td data-sheets-formula="=R477C[63]" data-sheets-value='{"1":3,"3":90}'>90</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Factoid"}'>Factoid</td> |
| | <td data-sheets-formula="=R477C[66]" data-sheets-value='{"1":3,"3":50}'>50</td> |
| | <td data-sheets-formula="=R477C[66]" data-sheets-value='{"1":3,"3":82.5}'>82.5</td> |
| | <td data-sheets-formula="=R477C[66]" data-sheets-value='{"1":3,"3":99}'>99</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Open quest"}'>Open quest</td> |
| | <td data-sheets-formula="=R477C[69]" data-sheets-value='{"1":3,"3":50}'>50</td> |
| | <td data-sheets-formula="=R477C[69]" data-sheets-value='{"1":3,"3":65}'>65</td> |
| | <td data-sheets-formula="=R477C[69]" data-sheets-value='{"1":3,"3":90}'>90</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Unkn quest"}'>Unkn quest</td> |
| | <td data-sheets-formula="=R477C[72]" data-sheets-value='{"1":3,"3":40}'>40</td> |
| | <td data-sheets-formula="=R477C[72]" data-sheets-value='{"1":3,"3":70}'>70</td> |
| | <td data-sheets-formula="=R477C[72]" data-sheets-value='{"1":3,"3":90}'>90</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Essay"}'>Essay</td> |
| | <td data-sheets-formula="=R477C[75]" data-sheets-value='{"1":3,"3":25}'>25</td> |
| | <td data-sheets-formula="=R477C[75]" data-sheets-value='{"1":3,"3":50}'>50</td> |
| | <td data-sheets-formula="=R477C[75]" data-sheets-value='{"1":3,"3":90}'>90</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Top forty"}'>Top forty</td> |
| | <td data-sheets-formula="=R477C[78]" data-sheets-value='{"1":3,"3":27.5}'>27.5</td> |
| | <td data-sheets-formula="=R477C[78]" data-sheets-value='{"1":3,"3":50}'>50</td> |
| | <td data-sheets-formula="=R477C[78]" data-sheets-value='{"1":3,"3":90}'>90</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Taylor"}'>Taylor</td> |
| | <td data-sheets-formula="=R477C[81]" data-sheets-value='{"1":3,"3":60}'>60</td> |
| | <td data-sheets-formula="=R477C[81]" data-sheets-value='{"1":3,"3":75}'>75</td> |
| | <td data-sheets-formula="=R477C[81]" data-sheets-value='{"1":3,"3":99}'>99</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Novel"}'>Novel</td> |
| | <td data-sheets-formula="=R477C[84]" data-sheets-value='{"1":3,"3":1}'>1</td> |
| | <td data-sheets-formula="=R477C[84]" data-sheets-value='{"1":3,"3":25}'>25</td> |
| | <td data-sheets-formula="=R477C[84]" data-sheets-value='{"1":3,"3":62.5}'>62.5</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Explain"}'>Explain</td> |
| | <td data-sheets-formula="=R477C[87]" data-sheets-value='{"1":3,"3":30}'>30</td> |
| | <td data-sheets-formula="=R477C[87]" data-sheets-value='{"1":3,"3":60}'>60</td> |
| | <td data-sheets-formula="=R477C[87]" data-sheets-value='{"1":3,"3":90}'>90</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Poker"}'>Poker</td> |
| | <td data-sheets-formula="=R477C[90]" data-sheets-value='{"1":3,"3":70}'>70</td> |
| | <td data-sheets-formula="=R477C[90]" data-sheets-value='{"1":3,"3":90}'>90</td> |
| | <td data-sheets-formula="=R477C[90]" data-sheets-value='{"1":3,"3":99}'>99</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Laws phys"}'>Laws phys</td> |
| | <td data-sheets-formula="=R477C[93]" data-sheets-value='{"1":3,"3":20}'>20</td> |
| | <td data-sheets-formula="=R477C[93]" data-sheets-value='{"1":3,"3":40}'>40</td> |
| | <td data-sheets-formula="=R477C[93]" data-sheets-value='{"1":3,"3":80}'>80</td> |
| | </tr> |
| | </tbody> |
| | </table> |
| | </HTML> |
| | |
| | |
| | = Fixed probabilities framing = |
| | |
| | |
| | |
| | |
| | <HTML> |
| | <p>Years by probability (medians)</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <table border="1" cellpadding="0" cellspacing="0" dir="ltr"> |
| | <colgroup> |
| | <col width="113"/> |
| | <col width="100"/> |
| | <col width="100"/> |
| | <col width="100"/> |
| | </colgroup> |
| | <tbody> |
| | <tr> |
| | <td> </td> |
| | <td data-sheets-value='{"1":2,"2":"10 percent"}'><strong>10 percent</strong></td> |
| | <td data-sheets-value='{"1":2,"2":"50 percent"}'><strong>50 percent</strong></td> |
| | <td data-sheets-value='{"1":2,"2":"90 percent"}'><strong>90 percent</strong></td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Rosetta"}'>Rosetta</td> |
| | <td data-sheets-formula="=R477C[0]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | <td data-sheets-formula="=R477C[0]" data-sheets-value='{"1":3,"3":20}'>20</td> |
| | <td data-sheets-formula="=R477C[0]" data-sheets-value='{"1":3,"3":50}'>50</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Subtitles"}'>Subtitles</td> |
| | <td data-sheets-formula="=R477C[3]" data-sheets-value='{"1":3,"3":5}'>5</td> |
| | <td data-sheets-formula="=R477C[3]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | <td data-sheets-formula="=R477C[3]" data-sheets-value='{"1":3,"3":15}'>15</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Translate"}'>Translate</td> |
| | <td data-sheets-formula="=R477C[6]" data-sheets-value='{"1":3,"3":3}'>3</td> |
| | <td data-sheets-formula="=R477C[6]" data-sheets-value='{"1":3,"3":7}'>7</td> |
| | <td data-sheets-formula="=R477C[6]" data-sheets-value='{"1":3,"3":15}'>15</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Phone bank"}'>Phone bank</td> |
| | <td data-sheets-formula="=R477C[9]" data-sheets-value='{"1":3,"3":3}'>3</td> |
| | <td data-sheets-formula="=R477C[9]" data-sheets-value='{"1":3,"3":6}'>6</td> |
| | <td data-sheets-formula="=R477C[9]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Class"}'>Class</td> |
| | <td data-sheets-formula="=R477C[12]" data-sheets-value='{"1":3,"3":2}'>2</td> |
| | <td data-sheets-formula="=R477C[12]" data-sheets-value='{"1":3,"3":4.5}'>4.5</td> |
| | <td data-sheets-formula="=R477C[12]" data-sheets-value='{"1":3,"3":6.5}'>6.5</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"One-shot"}'>One-shot</td> |
| | <td data-sheets-formula="=R477C[15]" data-sheets-value='{"1":3,"3":4.5}'>4.5</td> |
| | <td data-sheets-formula="=R477C[15]" data-sheets-value='{"1":3,"3":8}'>8</td> |
| | <td data-sheets-formula="=R477C[15]" data-sheets-value='{"1":3,"3":20}'>20</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Video scene"}'>Video scene</td> |
| | <td data-sheets-formula="=R477C[18]" data-sheets-value='{"1":3,"3":5}'>5</td> |
| | <td data-sheets-formula="=R477C[18]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | <td data-sheets-formula="=R477C[18]" data-sheets-value='{"1":3,"3":20}'>20</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Transcribe"}'>Transcribe</td> |
| | <td data-sheets-formula="=R477C[21]" data-sheets-value='{"1":3,"3":5}'>5</td> |
| | <td data-sheets-formula="=R477C[21]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | <td data-sheets-formula="=R477C[21]" data-sheets-value='{"1":3,"3":20}'>20</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Read aloud"}'>Read aloud</td> |
| | <td data-sheets-formula="=R477C[24]" data-sheets-value='{"1":3,"3":5}'>5</td> |
| | <td data-sheets-formula="=R477C[24]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | <td data-sheets-formula="=R477C[24]" data-sheets-value='{"1":3,"3":15}'>15</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Theorems"}'>Theorems</td> |
| | <td data-sheets-formula="=R477C[27]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | <td data-sheets-formula="=R477C[27]" data-sheets-value='{"1":3,"3":50}'>50</td> |
| | <td data-sheets-formula="=R477C[27]" data-sheets-value='{"1":3,"3":90}'>90</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Putnam"}'>Putnam</td> |
| | <td data-sheets-formula="=R477C[30]" data-sheets-value='{"1":3,"3":15}'>15</td> |
| | <td data-sheets-formula="=R477C[30]" data-sheets-value='{"1":3,"3":35}'>35</td> |
| | <td data-sheets-formula="=R477C[30]" data-sheets-value='{"1":3,"3":55}'>55</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Go low"}'>Go low</td> |
| | <td data-sheets-formula="=R477C[33]" data-sheets-value='{"1":3,"3":3.5}'>3.5</td> |
| | <td data-sheets-formula="=R477C[33]" data-sheets-value='{"1":3,"3":8.5}'>8.5</td> |
| | <td data-sheets-formula="=R477C[33]" data-sheets-value='{"1":3,"3":19.5}'>19.5</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Starcraft"}'>Starcraft</td> |
| | <td data-sheets-formula="=R477C[36]" data-sheets-value='{"1":3,"3":2}'>2</td> |
| | <td data-sheets-formula="=R477C[36]" data-sheets-value='{"1":3,"3":5}'>5</td> |
| | <td data-sheets-formula="=R477C[36]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Rand game"}'>Rand game</td> |
| | <td data-sheets-formula="=R477C[39]" data-sheets-value='{"1":3,"3":5}'>5</td> |
| | <td data-sheets-formula="=R477C[39]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | <td data-sheets-formula="=R477C[39]" data-sheets-value='{"1":3,"3":15}'>15</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Angry birds"}'>Angry birds</td> |
| | <td data-sheets-formula="=R477C[42]" data-sheets-value='{"1":3,"3":2}'>2</td> |
| | <td data-sheets-formula="=R477C[42]" data-sheets-value='{"1":3,"3":4}'>4</td> |
| | <td data-sheets-formula="=R477C[42]" data-sheets-value='{"1":3,"3":6}'>6</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Atari"}'>Atari</td> |
| | <td data-sheets-formula="=R477C[45]" data-sheets-value='{"1":3,"3":5}'>5</td> |
| | <td data-sheets-formula="=R477C[45]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | <td data-sheets-formula="=R477C[45]" data-sheets-value='{"1":3,"3":15}'>15</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Atari fifty"}'>Atari fifty</td> |
| | <td data-sheets-formula="=R477C[48]" data-sheets-value='{"1":3,"3":2}'>2</td> |
| | <td data-sheets-formula="=R477C[48]" data-sheets-value='{"1":3,"3":5}'>5</td> |
| | <td data-sheets-formula="=R477C[48]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Laundry"}'>Laundry</td> |
| | <td data-sheets-formula="=R477C[51]" data-sheets-value='{"1":3,"3":2}'>2</td> |
| | <td data-sheets-formula="=R477C[51]" data-sheets-value='{"1":3,"3":5.5}'>5.5</td> |
| | <td data-sheets-formula="=R477C[51]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Race"}'>Race</td> |
| | <td data-sheets-formula="=R477C[54]" data-sheets-value='{"1":3,"3":5}'>5</td> |
| | <td data-sheets-formula="=R477C[54]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | <td data-sheets-formula="=R477C[54]" data-sheets-value='{"1":3,"3":20}'>20</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Lego"}'>Lego</td> |
| | <td data-sheets-formula="=R477C[57]" data-sheets-value='{"1":3,"3":5}'>5</td> |
| | <td data-sheets-formula="=R477C[57]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | <td data-sheets-formula="=R477C[57]" data-sheets-value='{"1":3,"3":15}'>15</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Sort"}'>Sort</td> |
| | <td data-sheets-formula="=R477C[60]" data-sheets-value='{"1":3,"3":3}'>3</td> |
| | <td data-sheets-formula="=R477C[60]" data-sheets-value='{"1":3,"3":5}'>5</td> |
| | <td data-sheets-formula="=R477C[60]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Python"}'>Python</td> |
| | <td data-sheets-formula="=R477C[63]" data-sheets-value='{"1":3,"3":3}'>3</td> |
| | <td data-sheets-formula="=R477C[63]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | <td data-sheets-formula="=R477C[63]" data-sheets-value='{"1":3,"3":20}'>20</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Factoid"}'>Factoid</td> |
| | <td data-sheets-formula="=R477C[66]" data-sheets-value='{"1":3,"3":3}'>3</td> |
| | <td data-sheets-formula="=R477C[66]" data-sheets-value='{"1":3,"3":5}'>5</td> |
| | <td data-sheets-formula="=R477C[66]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Open quest"}'>Open quest</td> |
| | <td data-sheets-formula="=R477C[69]" data-sheets-value='{"1":3,"3":5}'>5</td> |
| | <td data-sheets-formula="=R477C[69]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | <td data-sheets-formula="=R477C[69]" data-sheets-value='{"1":3,"3":15}'>15</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Unkn quest"}'>Unkn quest</td> |
| | <td data-sheets-formula="=R477C[72]" data-sheets-value='{"1":3,"3":4}'>4</td> |
| | <td data-sheets-formula="=R477C[72]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | <td data-sheets-formula="=R477C[72]" data-sheets-value='{"1":3,"3":17.5}'>17.5</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Essay"}'>Essay</td> |
| | <td data-sheets-formula="=R477C[75]" data-sheets-value='{"1":3,"3":2}'>2</td> |
| | <td data-sheets-formula="=R477C[75]" data-sheets-value='{"1":3,"3":7}'>7</td> |
| | <td data-sheets-formula="=R477C[75]" data-sheets-value='{"1":3,"3":15}'>15</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Top forty"}'>Top forty</td> |
| | <td data-sheets-formula="=R477C[78]" data-sheets-value='{"1":3,"3":5}'>5</td> |
| | <td data-sheets-formula="=R477C[78]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | <td data-sheets-formula="=R477C[78]" data-sheets-value='{"1":3,"3":20}'>20</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Taylor"}'>Taylor</td> |
| | <td data-sheets-formula="=R477C[81]" data-sheets-value='{"1":3,"3":5}'>5</td> |
| | <td data-sheets-formula="=R477C[81]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | <td data-sheets-formula="=R477C[81]" data-sheets-value='{"1":3,"3":20}'>20</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Novel"}'>Novel</td> |
| | <td data-sheets-formula="=R477C[84]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | <td data-sheets-formula="=R477C[84]" data-sheets-value='{"1":3,"3":30}'>30</td> |
| | <td data-sheets-formula="=R477C[84]" data-sheets-value='{"1":3,"3":50}'>50</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Explain"}'>Explain</td> |
| | <td data-sheets-formula="=R477C[87]" data-sheets-value='{"1":3,"3":5}'>5</td> |
| | <td data-sheets-formula="=R477C[87]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | <td data-sheets-formula="=R477C[87]" data-sheets-value='{"1":3,"3":15}'>15</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Poker"}'>Poker</td> |
| | <td data-sheets-formula="=R477C[90]" data-sheets-value='{"1":3,"3":1}'>1</td> |
| | <td data-sheets-formula="=R477C[90]" data-sheets-value='{"1":3,"3":3}'>3</td> |
| | <td data-sheets-formula="=R477C[90]" data-sheets-value='{"1":3,"3":5.5}'>5.5</td> |
| | </tr> |
| | <tr> |
| | <td data-sheets-value='{"1":2,"2":"Laws phys"}'>Laws phys</td> |
| | <td data-sheets-formula="=R477C[93]" data-sheets-value='{"1":3,"3":5}'>5</td> |
| | <td data-sheets-formula="=R477C[93]" data-sheets-value='{"1":3,"3":10}'>10</td> |
| | <td data-sheets-formula="=R477C[93]" data-sheets-value='{"1":3,"3":20}'>20</td> |
| | </tr> |
| | </tbody> |
| | </table> |
| | </HTML> |
| | |
| | |
| | === Safety === |
| | |
| | |
| | == Stuart Russell’s problem == |
| | |
| | |
| | = Question = |
| | |
| | |
| | <HTML> |
| | <p>Participants were asked:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">Stuart Russell summarizes an argument for why highly advanced AI might pose a risk as follows:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 70px;"><em>The primary concern [with highly advanced AI] is not spooky emergent consciousness but simply the ability to make high-quality decisions. Here, quality refers to the expected outcome utility of actions taken […]. Now we have a problem:</em></p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 100px;"><em>1. The utility function may not be perfectly aligned with the values of the human race, which are (at best) very difficult to pin down.</em></p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 100px;"><em>2. Any sufficiently capable intelligent system will prefer to ensure its own continued existence and to acquire physical and computational resources – not for their own sake, but to succeed in its assigned task.</em></p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 70px;"><em>A system that is optimizing a function of n variables, where the objective depends on a subset of size k<n, will often set the remaining unconstrained variables to extreme values; if one of those unconstrained variables is actually something we care about, the solution found may be highly undesirable. This is essentially the old story of the genie in the lamp, or the sorcerer’s apprentice, or King Midas: you get exactly what you ask for, not what you want.</em></p> |
| | </HTML> |
| | |
| | |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">Do you think this argument points at an important problem?</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <ul> |
| | <li><div class="li"> |
| | <ul> |
| | <li><div class="li">No, not a real problem.</div></li> |
| | <li><div class="li">No, not an important problem.</div></li> |
| | <li><div class="li">Yes, a moderately important problem.</div></li> |
| | <li><div class="li">Yes, a very important problem.</div></li> |
| | <li><div class="li">Yes, among the most important problems in the field.</div></li> |
| | </ul> |
| | </div></li> |
| | </ul> |
| | </HTML> |
| | |
| | |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">How valuable is it to work on this problem <em><strong>today</strong></em>, compared to other problems in AI?</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <ul> |
| | <li><div class="li"> |
| | <ul> |
| | <li><div class="li">Much less valuable</div></li> |
| | <li><div class="li">Less valuable</div></li> |
| | <li><div class="li">As valuable as other problems</div></li> |
| | <li><div class="li">More valuable</div></li> |
| | <li><div class="li">Much more valuable</div></li> |
| | </ul> |
| | </div></li> |
| | </ul> |
| | </HTML> |
| | |
| | |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">How hard do you think this problem is compared to other problems in AI?</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <ul> |
| | <li><div class="li"> |
| | <ul> |
| | <li><div class="li">Much easier</div></li> |
| | <li><div class="li">Easier</div></li> |
| | <li><div class="li">As hard as other problems</div></li> |
| | <li><div class="li">Harder</div></li> |
| | <li><div class="li">Much harder</div></li> |
| | </ul> |
| | </div></li> |
| | </ul> |
| | </HTML> |
| | |
| | |
| | = Answers = |
| | |
| | |
| | |
| | |
| | == General safety == |
| | |
| | |
| | = Question = |
| | |
| | |
| | <HTML> |
| | <p>Participants were asked:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">Let ‘<strong>AI safety research</strong>’ include any AI-related research that, rather than being primarily aimed at improving the <em>capabilities</em> of AI systems, is instead primarily aimed at <em>minimizing potential risks</em> of AI systems (beyond what is already accomplished for those goals by increasing AI system capabilities).</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">Examples of AI safety research might include:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <ul> |
| | <li><div class="li"> |
| | <ul> |
| | <li><div class="li">Improving the human-interpretability of machine learning algorithms for the purpose of improving the safety and robustness of AI systems, not focused on improving AI capabilities</div></li> |
| | <li><div class="li">Research on long-term existential risks from AI systems</div></li> |
| | <li><div class="li">AI-specific formal verification research</div></li> |
| | <li><div class="li">Policy research about how to maximize the public benefits of AI</div></li> |
| | </ul> |
| | </div></li> |
| | </ul> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p style="padding-left: 40px;">How much should society prioritize <strong>AI safety research</strong>, relative to how much it is currently prioritized?</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <ul> |
| | <li><div class="li"> |
| | <ul> |
| | <li><div class="li">Much less</div></li> |
| | <li><div class="li">Less</div></li> |
| | <li><div class="li">About the same</div></li> |
| | <li><div class="li">More</div></li> |
| | <li><div class="li">Much more</div></li> |
| | </ul> |
| | </div></li> |
| | </ul> |
| | </HTML> |
| | |
| | |
| | = Answers = |
| | |
| | |
| | |
| | |
| | ==== Paper ==== |
| | |
| | |
| | <HTML> |
| | <p>Some results are reported in <em>When Will AI Exceed Human Performance? Evidence from AI Experts</em><span class="easy-footnote-margin-adjust" id="easy-footnote-3-753"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-3-753" title='Grace, Katja, John Salvatier, Allan Dafoe, Baobao Zhang, and Owain Evans. “When Will AI Exceed Human Performance? Evidence from AI Experts.” <i>ArXiv:1705.08807 [Cs]</i>, May 3, 2018. <a href="http://arxiv.org/abs/1705.08807">http://arxiv.org/abs/1705.08807</a>.<span style="font-size: inherit;">'><sup>3</sup></a></span>.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>Some notes on interpreting the paper<span class="easy-footnote-margin-adjust" id="easy-footnote-4-753"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-4-753" title='Originally in an AI Impacts blog post:</p> <p>AI Impacts. “Some Survey Results!,” June 8, 2017. <a href="https://aiimpacts.org/some-survey-results/">https://aiimpacts.org/some-survey-results/</a>.'><sup>4</sup></a></span>:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <ol> |
| | <li><div class="li">The milestones in the timeline and in the abstract are from three different sets of questions. There seems to be a large framing effect between two of them—full automation of labor is logically required to be before HLMI, and yet it is predicted much later—and it is unclear whether people answer the third set of questions (about narrow tasks) more like the one about HLMI or more like the one about occupations. Plus even if there were no framing effect to worry about, we should expect milestones about narrow tasks to be much earlier than milestones about very similar sounding occupations. For instance, if there were an <span class="contentItalic">o</span><span class="contentItalic">c</span><span class="contentItalic">c</span><span class="contentItalic">u</span><span class="contentItalic">p</span><span class="contentItalic">a</span><span class="contentItalic">t</span><span class="contentItalic">i</span><span class="contentItalic">o</span><span class="contentItalic">n</span> ‘math researcher’, it should be later than the narrow task summarized here as ‘math research’. So there is a risk of interpreting the figure as saying AI research is harder than math research, when really the ‘-er’ is all-important. So to help avoid confusion, here is the timeline colored in by which set of questions each milestone came from. The blue one was asked on its own. The orange ones were always asked together: first all four occupations, then they were asked for an occupation they expected to be very late, and when they expected it, then full automation of labor. The pink milestones were randomized, so that each person got four. There are a lot more pink milestones not included here, but included in the long table at the end of the paper.<a href="http://aiimpacts.org/wp-content/uploads/2017/06/timelineofairesults.jpeg"><img alt="" class="size-full wp-image-844" height="1302" loading="lazy" sizes="(max-width: 1392px) 100vw, 1392px" src="https://aiimpacts.org/wp-content/uploads/2017/06/timelineofairesults.jpeg" srcset="https://aiimpacts.org/wp-content/uploads/2017/06/timelineofairesults.jpeg 1392w, https://aiimpacts.org/wp-content/uploads/2017/06/timelineofairesults-300x281.jpeg 300w, https://aiimpacts.org/wp-content/uploads/2017/06/timelineofairesults-768x718.jpeg 768w, https://aiimpacts.org/wp-content/uploads/2017/06/timelineofairesults-1024x958.jpeg 1024w" width="1392"/></a> |
| | </div></li> |
| | <li><div class="li">In Figure 2 and Table S5 I believe the word ‘median’ means we are talking about the ‘50% chance of occurring’ number, and the dates given are this ‘median’ (50% chance) date for a distribution that was made by averaging together all of the different people’s distributions (or what we guess their distributions are like from three data points).</div></li> |
| | </ol> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p><br/> |
| | <br/> |
| | <em>Authors: Katja Grace, John Salvatier, Allan Dafoe, Baobao Zhang, and Owain Evans. This page is an extended analyses of research published in Grace et al 2017</em> <span class="easy-footnote-margin-adjust" id="easy-footnote-5-753"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-5-753" title='Grace, Katja, John Salvatier, Allan Dafoe, Baobao Zhang, and Owain Evans. “When Will AI Exceed Human Performance? Evidence from AI Experts.” ArXiv:1705.08807 [Cs], 2017. <a href="http://arxiv.org/abs/1705.08807">http://arxiv.org/abs/1705.08807</a>.'><sup>5</sup></a></span><em>, so includes research contributions from all authors, but Katja compiled and described these, and added additional figures and analyses, so composition decisions and opinion reflect her views and not necessarily those of the group, and much of the specific analysis has not been vetted by the group in this form.</em></p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>Suggested citation:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>Grace, Katja, John Salvatier, Allan Dafoe, Baobao Zhang, and Owain Evans. “2016 Expert Survey on Progress in AI.” In <em>AI Impacts</em>, December 14, 2016. <a href="/doku.php?id=ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:2016_expert_survey_on_progress_in_ai">https://aiimpacts.org/2016-expert-survey-on-progress-in-ai/</a>.</p> |
| | </HTML> |
| | |
| | |
| | ===== Notes ===== |
| | |
| | |
| | <HTML> |
| | <ol class="easy-footnotes-wrapper"> |
| | <li><div class="li"> |
| | <span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-1-753"></span>A small number of respondents may have answered a slightly different version of this question in an initial round, in which case those answers are not included here.<a class="easy-footnote-to-top" href="#easy-footnote-1-753"></a> |
| | </div></li> |
| | <li><div class="li"> |
| | <span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-2-753"></span>One person has categorized these responses, and another has checked and corroborated their categorizations.<a class="easy-footnote-to-top" href="#easy-footnote-2-753"></a> |
| | </div></li> |
| | <li><div class="li"> |
| | <span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-3-753"></span>Grace, Katja, John Salvatier, Allan Dafoe, Baobao Zhang, and Owain Evans. “When Will AI Exceed Human Performance? Evidence from AI Experts.” <i>ArXiv:1705.08807 [Cs]</i>, May 3, 2018. <a href="http://arxiv.org/abs/1705.08807">http://arxiv.org/abs/1705.08807</a>.<span style="font-size: inherit"><a class="easy-footnote-to-top" href="#easy-footnote-3-753"></a></span> |
| | </div></li> |
| | <li><div class="li"> |
| | <span style="font-size: inherit"><span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-4-753"></span>Originally in an AI Impacts blog post:<br/> |
| | <br/></span> |
| | <p>AI Impacts. “Some Survey Results!,” June 8, 2017. <a href="https://aiimpacts.org/some-survey-results/">https://aiimpacts.org/some-survey-results/</a>.<a class="easy-footnote-to-top" href="#easy-footnote-4-753"></a></p> |
| | </div></li> |
| | <li><div class="li"> |
| | <span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-5-753"></span>Grace, Katja, John Salvatier, Allan Dafoe, Baobao Zhang, and Owain Evans. “When Will AI Exceed Human Performance? Evidence from AI Experts.” ArXiv:1705.08807 [Cs], 2017. <a href="http://arxiv.org/abs/1705.08807">http://arxiv.org/abs/1705.08807</a>.<a class="easy-footnote-to-top" href="#easy-footnote-5-753"></a> |
| | </div></li> |
| | </ol> |
| | </HTML> |
| | |
| | |
| |
