https://wiki.aiimpacts.org/ 2026-04-29T16:03:22+00:00 FeedCreator 1.8 (info@mypapit.net) 2024-07-06T03:32:23+00:00 2024-07-06T03:32:23+00:00 https://wiki.aiimpacts.org/speed_of_ai_transition/range_of_human_performance/the_range_of_human_intelligence?rev=1720236743&do=diff Anonymous anonymous@undisclosed.example.com <pre> @@ -221,13 +221,10 @@ &lt;HTML&gt; &lt;ol class=&quot;easy-footnotes-wrapper&quot;&gt; &lt;li&gt;&lt;div class=&quot;li&quot;&gt; &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-1-191&quot;&gt;&lt;/span&gt;“My Childhood Role Model – LessWrong 2.0.” Accessed June 3, 2020. &lt;a href=&quot;https://www.lesswrong.com/posts/3Jpchgy53D2gB5qdk/my-childhood-role-model&quot;&gt;https://www.lesswrong.com/posts/3Jpchgy53D2gB5qdk/my-childhood-role-model&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-1-191&quot;&gt;&lt;/a&gt; - &lt;/div&gt;&lt;/li&gt; - &lt;li&gt;&lt;div class=&quot;li&quot;&gt; - &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-2-191&quot;&gt;&lt;/span&gt;In particular&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-2-191&quot;&gt;&lt;/a&gt; &lt;/div&gt;&lt;/li&gt; &lt;/ol&gt; &lt;/HTML&gt; </pre>

<pre> @@ -221,13 +221,10 @@ &lt;HTML&gt; &lt;ol class=&quot;easy-footnotes-wrapper&quot;&gt; &lt;li&gt;&lt;div class=&quot;li&quot;&gt; &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-1-191&quot;&gt;&lt;/span&gt;“My Childhood Role Model – LessWrong 2.0.” Accessed June 3, 2020. &lt;a href=&quot;https://www.lesswrong.com/posts/3Jpchgy53D2gB5qdk/my-childhood-role-model&quot;&gt;https://www.lesswrong.com/posts/3Jpchgy53D2gB5qdk/my-childhood-role-model&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-1-191&quot;&gt;&lt;/a&gt; - &lt;/div&gt;&lt;/li&gt; - &lt;li&gt;&lt;div class=&quot;li&quot;&gt; - &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-2-191&quot;&gt;&lt;/span&gt;In particular&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-2-191&quot;&gt;&lt;/a&gt; &lt;/div&gt;&lt;/li&gt; &lt;/ol&gt; &lt;/HTML&gt; </pre>

2022-11-02T19:02:30+00:00 2022-11-02T19:02:30+00:00 https://wiki.aiimpacts.org/speed_of_ai_transition/range_of_human_performance/time_for_ai_to_cross_the_human_performance_range_in_chess?rev=1667415750&do=diff Anonymous anonymous@undisclosed.example.com <pre> @@ -27,9 +27,9 @@ === Beginner to superhuman range === We could not find transparent sources for low computer chess Elo records, but it seems common to place Elo scores of 800-1200 in the 1950s and 1960s. In his book Robot (1999)((Moravec, Hans. Robot: Mere Machine to Transcendent Mind, n.d., p71, also at https://frc.ri.cmu.edu/~hpm/book97/ch3/index.html)), Moravec gives the diagram shown in Figure 1, which puts a machine with an Elo of around 800 in 1957. He does not appear to provide a source for this however. Figure 2 shows another figure without sources from a 2002 article by L. Stephen Coles at Dr. Dobbs ((“Computer Chess: The Drosophila of AI | Dr Dobb’s.” Accessed October 13, 2020. https://www.drdobbs.com/parallel/computer-chess-the-drosophila-of-ai/184405171?pgno=2. p2)), which puts some machine at over 1000 in around 1950. To err on the side of assuming narrow human ranges, and because Moravec appears to be a more reliable source, we use his data here. This means that machine chess performance entered the human range in 1957 at the latest. - [{{:speed_of_ai_transition:range_of_human_performance:chess.plot.150.jpg?600|Figure 1: Graph from Moravec, 1999((Moravec, Hans. Robot: Mere Machine to Transcendent Mind, n.d., p71, also at https://frc.ri.cmu.edu/~hpm/book97/ch3/index.html)). We did not find Moravec’s sources for these numbers.}}] + [{{:speed_of_ai_transition:range_of_human_performance:chess.plot.150.jpg?600|Figure 1: Graph from Moravec, 1999((Moravec, Hans. Robot: Mere Machine to Transcendent Mind, n.d., p71, also at https://frc.ri.cmu.edu/~hpm/book97/ch3/index.html)). We did not find Moravec’s sources for these numbers.}}]\\ [{{:speed_of_ai_transition:range_of_human_performance:chess_progress_source_unknown.gif?600|Figure 2: Chess AI progress compared to human performance, from Coles 2002((“Computer Chess: The Drosophila of AI | Dr Dobb’s.” Accessed October 13, 2020. https://www.drdobbs.com/parallel/computer-chess-the-drosophila-of-ai/184405171?pgno=2. p2)). The image appears to confusingly claim that the present time is around 1993, in which case the right of the graph (after ‘now’) must be imagined, though it appears to be approximately correct.}}] The chess computer Deep Blue famously beat the then world-champion Kasparov under tournament conditions in 1997.((“Deep Blue versus Garry Kasparov.” In Wikipedia, October 1, 2020. https://en.wikipedia.org/w/index.php?title=Deep_Blue_versus_Garry_Kasparov&amp;oldid=981337286.)) However this does not imply that Deep Blue was at that point overall more capable than Kasparov, i.e. had a higher Elo rating.((A group of Redditors discuss:\\ </pre>

<pre> @@ -27,9 +27,9 @@ === Beginner to superhuman range === We could not find transparent sources for low computer chess Elo records, but it seems common to place Elo scores of 800-1200 in the 1950s and 1960s. In his book Robot (1999)((Moravec, Hans. Robot: Mere Machine to Transcendent Mind, n.d., p71, also at https://frc.ri.cmu.edu/~hpm/book97/ch3/index.html)), Moravec gives the diagram shown in Figure 1, which puts a machine with an Elo of around 800 in 1957. He does not appear to provide a source for this however. Figure 2 shows another figure without sources from a 2002 article by L. Stephen Coles at Dr. Dobbs ((“Computer Chess: The Drosophila of AI | Dr Dobb’s.” Accessed October 13, 2020. https://www.drdobbs.com/parallel/computer-chess-the-drosophila-of-ai/184405171?pgno=2. p2)), which puts some machine at over 1000 in around 1950. To err on the side of assuming narrow human ranges, and because Moravec appears to be a more reliable source, we use his data here. This means that machine chess performance entered the human range in 1957 at the latest. - [{{:speed_of_ai_transition:range_of_human_performance:chess.plot.150.jpg?600|Figure 1: Graph from Moravec, 1999((Moravec, Hans. Robot: Mere Machine to Transcendent Mind, n.d., p71, also at https://frc.ri.cmu.edu/~hpm/book97/ch3/index.html)). We did not find Moravec’s sources for these numbers.}}] + [{{:speed_of_ai_transition:range_of_human_performance:chess.plot.150.jpg?600|Figure 1: Graph from Moravec, 1999((Moravec, Hans. Robot: Mere Machine to Transcendent Mind, n.d., p71, also at https://frc.ri.cmu.edu/~hpm/book97/ch3/index.html)). We did not find Moravec’s sources for these numbers.}}]\\ [{{:speed_of_ai_transition:range_of_human_performance:chess_progress_source_unknown.gif?600|Figure 2: Chess AI progress compared to human performance, from Coles 2002((“Computer Chess: The Drosophila of AI | Dr Dobb’s.” Accessed October 13, 2020. https://www.drdobbs.com/parallel/computer-chess-the-drosophila-of-ai/184405171?pgno=2. p2)). The image appears to confusingly claim that the present time is around 1993, in which case the right of the graph (after ‘now’) must be imagined, though it appears to be approximately correct.}}] The chess computer Deep Blue famously beat the then world-champion Kasparov under tournament conditions in 1997.((“Deep Blue versus Garry Kasparov.” In Wikipedia, October 1, 2020. https://en.wikipedia.org/w/index.php?title=Deep_Blue_versus_Garry_Kasparov&amp;oldid=981337286.)) However this does not imply that Deep Blue was at that point overall more capable than Kasparov, i.e. had a higher Elo rating.((A group of Redditors discuss:\\ </pre>

2022-09-21T07:37:41+00:00 2022-09-21T07:37:41+00:00 https://wiki.aiimpacts.org/speed_of_ai_transition/range_of_human_performance/time_for_ai_to_cross_the_human_performance_range_in_diabetic_retinopathy?rev=1663745861&do=diff Anonymous anonymous@undisclosed.example.com <pre> @@ -1 +1,87 @@ + ====== Time for AI to cross the human performance range in diabetic retinopathy ====== + + // Published 21 November, 2018; last updated 20 January, 2021 // + + &lt;HTML&gt; + &lt;p&gt;&lt;span style=&quot;font-weight: 400;&quot;&gt;In diabetic retinopathy,&lt;/span&gt; automated systems &lt;span style=&quot;font-weight: 400;&quot;&gt;started out just below expert human level performance, and took around ten years to reach expert human level performance.&lt;/span&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + + ===== Details ===== + + + &lt;HTML&gt; + &lt;p&gt;&lt;span style=&quot;font-weight: 400;&quot;&gt;Diabetic retinopathy is a complication of diabetes in which the back of the eye is damaged by high blood sugar levels.&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-1-1241&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-1-1241&quot; title=&#039; See e.g. &amp;lt;a href=&quot;https://www.nhs.uk/conditions/diabetic-retinopathy/&quot;&amp;gt;https://www.nhs.uk/conditions/diabetic-retinopathy/&amp;lt;/a&amp;gt; &#039;&gt;&lt;sup&gt;1&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; It is the most common cause of blindness among working-age adults.&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-2-1241&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-2-1241&quot; title=&#039;See &amp;lt;a href=&quot;https://nei.nih.gov/health/diabetic/retinopathy&quot;&amp;gt;https://nei.nih.gov/health/diabetic/retinopathy&amp;lt;/a&amp;gt;&#039;&gt;&lt;sup&gt;2&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; The disease is diagnosed by examining images of the back of the eye.&lt;/span&gt; &lt;span style=&quot;font-weight: 400;&quot;&gt;The gold standard used for diabetic retinopathy diagnosis is typically some sort of pooling mechanism over several expert opinions. Thus, in the papers below, each time expert sensitivity/specificity (Se/Sp) is considered, it is the Se/Sp of individual experts graded against aggregate expert agreement.&lt;/span&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;p&gt;&lt;span style=&quot;font-weight: 400;&quot;&gt;As a rough benchmark for expert-level performance we’ll take the average Se/Sp of ophthalmologists from a few studies. Based on Google Brain’s work (detailed below), this paper &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-3-1241&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-3-1241&quot; title=&#039; See Results section before adjudication and consensus &amp;lt;/span&amp;gt;&amp;lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/23494039&quot;&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt;https://www.ncbi.nlm.nih.gov/pubmed/23494039&amp;lt;/span&amp;gt;&amp;lt;/a&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt; &#039;&gt;&lt;sup&gt;3&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;, and this paper &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-4-1241&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-4-1241&quot; title=&#039; See Figure 3 &amp;lt;/span&amp;gt;&amp;lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2911785/&quot;&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2911785/&amp;lt;/span&amp;gt;&amp;lt;/a&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt; &#039;&gt;&lt;sup&gt;4&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; , the average specificity of 14 opthamologists, which indicates expert human-level performance, is 95% and the average sensitivity is 82%.&lt;/span&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;p&gt;&lt;span style=&quot;font-weight: 400;&quot;&gt;As far as we can tell, 1996 is when the first algorithm automatically detecting diabetic retinopathy was developed. When compared to opthamologists’ ratings, the algorithm achieved 88.4% sensitivity and 83.5% specificity.&lt;/span&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;p&gt;&lt;span style=&quot;font-weight: 400;&quot;&gt;In late 2016 Google algorithms were on par with eight opthamologist diagnoses of diabetic retinopathy. See Figure 1.&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-5-1241&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-5-1241&quot; title=&#039; &amp;lt;a href=&quot;https://ai.googleblog.com/2016/11/deep-learning-for-detection-of-diabetic.html&quot;&amp;gt;https://ai.googleblog.com/2016/11/deep-learning-for-detection-of-diabetic.html&amp;lt;/a&amp;gt; &#039;&gt;&lt;sup&gt;5&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;font-weight: 400;&quot;&gt; The high-sensitivity operating point (labelled on the graph) achieved 97.5/93.4 Se/Sp.   &lt;/span&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;figure aria-describedby=&quot;caption-attachment-1242&quot; class=&quot;wp-caption alignnone&quot; id=&quot;attachment_1242&quot; style=&quot;width: 553px&quot;&gt; + &lt;a href=&quot;http://aiimpacts.org/wp-content/uploads/2018/11/eyepacs1.png&quot;&gt;&lt;img alt=&quot;&quot; class=&quot;wp-image-1242 size-full&quot; height=&quot;550&quot; sizes=&quot;(max-width: 553px) 100vw, 553px&quot; src=&quot;https://aiimpacts.org/wp-content/uploads/2018/11/eyepacs1.png&quot; srcset=&quot;https://aiimpacts.org/wp-content/uploads/2018/11/eyepacs1.png 553w, https://aiimpacts.org/wp-content/uploads/2018/11/eyepacs1-150x150.png 150w, https://aiimpacts.org/wp-content/uploads/2018/11/eyepacs1-300x298.png 300w&quot; width=&quot;553&quot;/&gt;&lt;/a&gt; + &lt;figcaption class=&quot;wp-caption-text&quot; id=&quot;caption-attachment-1242&quot;&gt; + Figure 1: Performance comparison of a late 2016 Google algorithm, and eight opthalmologists, from here.&lt;span style=&quot;font-weight: 400;&quot;&gt; The black curve represents the algorithm and the eight colored dots are opthamologists.&lt;/span&gt; + &lt;/figcaption&gt; + &lt;/figure&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;p&gt;&lt;span style=&quot;font-weight: 400;&quot;&gt;Many other papers were published in between 1996 and 2016. However, none of them achieved better than expert human-level performance on both specificity and sensitivity. For instance 86/77 Se/Sp was achieved in 2007, 97/59 in 2013, and 94/72 by another team in 2016. &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-6-1241&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-6-1241&quot; title=&#039; ‘&amp;lt;/span&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt;Automated and semi-automated diabetic retinopathy evaluation has been previously studied by other groups. Abràmoff et al&amp;lt;/span&amp;gt;&amp;lt;a href=&quot;https://jamanetwork.com/journals/jama/fullarticle/2588763#joi160132r4&quot;&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt;4&amp;lt;/span&amp;gt;&amp;lt;/a&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt; reported a sensitivity of 96.8% at a specificity of 59.4% for detecting referable diabetic retinopathy on the publicly available Messidor-2 data set.&amp;lt;/span&amp;gt;&amp;lt;a href=&quot;https://jamanetwork.com/journals/jama/fullarticle/2588763#joi160132r9&quot;&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt;9&amp;lt;/span&amp;gt;&amp;lt;/a&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt;Solanki et al&amp;lt;/span&amp;gt;&amp;lt;a href=&quot;https://jamanetwork.com/journals/jama/fullarticle/2588763#joi160132r12&quot;&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt;12&amp;lt;/span&amp;gt;&amp;lt;/a&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt; reported a sensitivity of 93.8% at a specificity of 72.2% on the same data set. A study by Philip et al&amp;lt;/span&amp;gt;&amp;lt;a href=&quot;https://jamanetwork.com/journals/jama/fullarticle/2588763#joi160132r21&quot;&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt;21&amp;lt;/span&amp;gt;&amp;lt;/a&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt; reported a sensitivity of 86.2% at a specificity of 76.8% for predicting disease vs no disease on their own data set of 14, 406 images.’ &amp;lt;/span&amp;gt;&amp;lt;a href=&quot;https://jamanetwork.com/journals/jama/fullarticle/2588763&quot;&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt;https://jamanetwork.com/journals/jama/fullarticle/2588763&amp;lt;/span&amp;gt;&amp;lt;/a&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt; &#039;&gt;&lt;sup&gt;6&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;&lt;/span&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;p&gt;&lt;span style=&quot;font-weight: 400;&quot;&gt;Thus it took about&lt;/span&gt; &lt;b&gt;ten years&lt;/b&gt; &lt;span style=&quot;font-weight: 400;&quot;&gt;to go from just below expert human level performance to slightly superhuman performance.&lt;/span&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + ===== Contributions ===== + + + &lt;HTML&gt; + &lt;p&gt;&lt;em&gt;Aysja Johnson researched and wrote this page. Justis Mills and Katja Grace contributed feedback.&lt;/em&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + ===== Footnotes ===== + + + &lt;HTML&gt; + &lt;ol class=&quot;easy-footnotes-wrapper&quot;&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-1-1241&quot;&gt;&lt;/span&gt; See e.g. &lt;a href=&quot;https://www.nhs.uk/conditions/diabetic-retinopathy/&quot;&gt;https://www.nhs.uk/conditions/diabetic-retinopathy/&lt;/a&gt; &lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-1-1241&quot;&gt;&lt;/a&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-2-1241&quot;&gt;&lt;/span&gt;See &lt;a href=&quot;https://nei.nih.gov/health/diabetic/retinopathy&quot;&gt;https://nei.nih.gov/health/diabetic/retinopathy&lt;/a&gt;&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-2-1241&quot;&gt;&lt;/a&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-3-1241&quot;&gt;&lt;/span&gt; See Results section before adjudication and consensus &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/23494039&quot;&gt;&lt;span style=&quot;font-weight: 400&quot;&gt;https://www.ncbi.nlm.nih.gov/pubmed/23494039&lt;/span&gt;&lt;/a&gt; &lt;span style=&quot;font-weight: 400&quot;&gt;&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-3-1241&quot;&gt;&lt;/a&gt;&lt;/span&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span style=&quot;font-weight: 400&quot;&gt;&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-4-1241&quot;&gt;&lt;/span&gt; See Figure 3&lt;/span&gt; &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2911785/&quot;&gt;&lt;span style=&quot;font-weight: 400&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2911785/&lt;/span&gt;&lt;/a&gt; &lt;span style=&quot;font-weight: 400&quot;&gt;&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-4-1241&quot;&gt;&lt;/a&gt;&lt;/span&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt;&lt;span style=&quot;font-weight: 400&quot;&gt;&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-5-1241&quot;&gt;&lt;/span&gt; &lt;a href=&quot;https://ai.googleblog.com/2016/11/deep-learning-for-detection-of-diabetic.html&quot;&gt;https://ai.googleblog.com/2016/11/deep-learning-for-detection-of-diabetic.html&lt;/a&gt; &lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-5-1241&quot;&gt;&lt;/a&gt;&lt;/span&gt;&lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-6-1241&quot;&gt;&lt;/span&gt; ‘&lt;span style=&quot;font-weight: 400&quot;&gt;Automated and semi-automated diabetic retinopathy evaluation has been previously studied by other groups. Abràmoff et al&lt;/span&gt;&lt;a href=&quot;https://jamanetwork.com/journals/jama/fullarticle/2588763#joi160132r4&quot;&gt;&lt;span style=&quot;font-weight: 400&quot;&gt;4&lt;/span&gt;&lt;/a&gt; &lt;span style=&quot;font-weight: 400&quot;&gt;reported a sensitivity of 96.8% at a specificity of 59.4% for detecting referable diabetic retinopathy on the publicly available Messidor-2 data set.&lt;/span&gt;&lt;a href=&quot;https://jamanetwork.com/journals/jama/fullarticle/2588763#joi160132r9&quot;&gt;&lt;span style=&quot;font-weight: 400&quot;&gt;9&lt;/span&gt;&lt;/a&gt;&lt;span style=&quot;font-weight: 400&quot;&gt;Solanki et al&lt;/span&gt;&lt;a href=&quot;https://jamanetwork.com/journals/jama/fullarticle/2588763#joi160132r12&quot;&gt;&lt;span style=&quot;font-weight: 400&quot;&gt;12&lt;/span&gt;&lt;/a&gt; &lt;span style=&quot;font-weight: 400&quot;&gt;reported a sensitivity of 93.8% at a specificity of 72.2% on the same data set. A study by Philip et al&lt;/span&gt;&lt;a href=&quot;https://jamanetwork.com/journals/jama/fullarticle/2588763#joi160132r21&quot;&gt;&lt;span style=&quot;font-weight: 400&quot;&gt;21&lt;/span&gt;&lt;/a&gt; &lt;span style=&quot;font-weight: 400&quot;&gt;reported a sensitivity of 86.2% at a specificity of 76.8% for predicting disease vs no disease on their own data set of 14, 406 images.’&lt;/span&gt; &lt;a href=&quot;https://jamanetwork.com/journals/jama/fullarticle/2588763&quot;&gt;&lt;span style=&quot;font-weight: 400&quot;&gt;https://jamanetwork.com/journals/jama/fullarticle/2588763&lt;/span&gt;&lt;/a&gt; &lt;span style=&quot;font-weight: 400&quot;&gt;&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-6-1241&quot;&gt;&lt;/a&gt;&lt;/span&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;/ol&gt; + &lt;/HTML&gt; + + </pre>

<pre> @@ -1 +1,87 @@ + ====== Time for AI to cross the human performance range in diabetic retinopathy ====== + + // Published 21 November, 2018; last updated 20 January, 2021 // + + &lt;HTML&gt; + &lt;p&gt;&lt;span style=&quot;font-weight: 400;&quot;&gt;In diabetic retinopathy,&lt;/span&gt; automated systems &lt;span style=&quot;font-weight: 400;&quot;&gt;started out just below expert human level performance, and took around ten years to reach expert human level performance.&lt;/span&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + + ===== Details ===== + + + &lt;HTML&gt; + &lt;p&gt;&lt;span style=&quot;font-weight: 400;&quot;&gt;Diabetic retinopathy is a complication of diabetes in which the back of the eye is damaged by high blood sugar levels.&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-1-1241&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-1-1241&quot; title=&#039; See e.g. &amp;lt;a href=&quot;https://www.nhs.uk/conditions/diabetic-retinopathy/&quot;&amp;gt;https://www.nhs.uk/conditions/diabetic-retinopathy/&amp;lt;/a&amp;gt; &#039;&gt;&lt;sup&gt;1&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; It is the most common cause of blindness among working-age adults.&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-2-1241&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-2-1241&quot; title=&#039;See &amp;lt;a href=&quot;https://nei.nih.gov/health/diabetic/retinopathy&quot;&amp;gt;https://nei.nih.gov/health/diabetic/retinopathy&amp;lt;/a&amp;gt;&#039;&gt;&lt;sup&gt;2&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; The disease is diagnosed by examining images of the back of the eye.&lt;/span&gt; &lt;span style=&quot;font-weight: 400;&quot;&gt;The gold standard used for diabetic retinopathy diagnosis is typically some sort of pooling mechanism over several expert opinions. Thus, in the papers below, each time expert sensitivity/specificity (Se/Sp) is considered, it is the Se/Sp of individual experts graded against aggregate expert agreement.&lt;/span&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;p&gt;&lt;span style=&quot;font-weight: 400;&quot;&gt;As a rough benchmark for expert-level performance we’ll take the average Se/Sp of ophthalmologists from a few studies. Based on Google Brain’s work (detailed below), this paper &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-3-1241&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-3-1241&quot; title=&#039; See Results section before adjudication and consensus &amp;lt;/span&amp;gt;&amp;lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/23494039&quot;&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt;https://www.ncbi.nlm.nih.gov/pubmed/23494039&amp;lt;/span&amp;gt;&amp;lt;/a&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt; &#039;&gt;&lt;sup&gt;3&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;, and this paper &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-4-1241&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-4-1241&quot; title=&#039; See Figure 3 &amp;lt;/span&amp;gt;&amp;lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2911785/&quot;&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2911785/&amp;lt;/span&amp;gt;&amp;lt;/a&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt; &#039;&gt;&lt;sup&gt;4&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; , the average specificity of 14 opthamologists, which indicates expert human-level performance, is 95% and the average sensitivity is 82%.&lt;/span&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;p&gt;&lt;span style=&quot;font-weight: 400;&quot;&gt;As far as we can tell, 1996 is when the first algorithm automatically detecting diabetic retinopathy was developed. When compared to opthamologists’ ratings, the algorithm achieved 88.4% sensitivity and 83.5% specificity.&lt;/span&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;p&gt;&lt;span style=&quot;font-weight: 400;&quot;&gt;In late 2016 Google algorithms were on par with eight opthamologist diagnoses of diabetic retinopathy. See Figure 1.&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-5-1241&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-5-1241&quot; title=&#039; &amp;lt;a href=&quot;https://ai.googleblog.com/2016/11/deep-learning-for-detection-of-diabetic.html&quot;&amp;gt;https://ai.googleblog.com/2016/11/deep-learning-for-detection-of-diabetic.html&amp;lt;/a&amp;gt; &#039;&gt;&lt;sup&gt;5&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;font-weight: 400;&quot;&gt; The high-sensitivity operating point (labelled on the graph) achieved 97.5/93.4 Se/Sp.   &lt;/span&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;figure aria-describedby=&quot;caption-attachment-1242&quot; class=&quot;wp-caption alignnone&quot; id=&quot;attachment_1242&quot; style=&quot;width: 553px&quot;&gt; + &lt;a href=&quot;http://aiimpacts.org/wp-content/uploads/2018/11/eyepacs1.png&quot;&gt;&lt;img alt=&quot;&quot; class=&quot;wp-image-1242 size-full&quot; height=&quot;550&quot; sizes=&quot;(max-width: 553px) 100vw, 553px&quot; src=&quot;https://aiimpacts.org/wp-content/uploads/2018/11/eyepacs1.png&quot; srcset=&quot;https://aiimpacts.org/wp-content/uploads/2018/11/eyepacs1.png 553w, https://aiimpacts.org/wp-content/uploads/2018/11/eyepacs1-150x150.png 150w, https://aiimpacts.org/wp-content/uploads/2018/11/eyepacs1-300x298.png 300w&quot; width=&quot;553&quot;/&gt;&lt;/a&gt; + &lt;figcaption class=&quot;wp-caption-text&quot; id=&quot;caption-attachment-1242&quot;&gt; + Figure 1: Performance comparison of a late 2016 Google algorithm, and eight opthalmologists, from here.&lt;span style=&quot;font-weight: 400;&quot;&gt; The black curve represents the algorithm and the eight colored dots are opthamologists.&lt;/span&gt; + &lt;/figcaption&gt; + &lt;/figure&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;p&gt;&lt;span style=&quot;font-weight: 400;&quot;&gt;Many other papers were published in between 1996 and 2016. However, none of them achieved better than expert human-level performance on both specificity and sensitivity. For instance 86/77 Se/Sp was achieved in 2007, 97/59 in 2013, and 94/72 by another team in 2016. &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-6-1241&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-6-1241&quot; title=&#039; ‘&amp;lt;/span&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt;Automated and semi-automated diabetic retinopathy evaluation has been previously studied by other groups. Abràmoff et al&amp;lt;/span&amp;gt;&amp;lt;a href=&quot;https://jamanetwork.com/journals/jama/fullarticle/2588763#joi160132r4&quot;&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt;4&amp;lt;/span&amp;gt;&amp;lt;/a&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt; reported a sensitivity of 96.8% at a specificity of 59.4% for detecting referable diabetic retinopathy on the publicly available Messidor-2 data set.&amp;lt;/span&amp;gt;&amp;lt;a href=&quot;https://jamanetwork.com/journals/jama/fullarticle/2588763#joi160132r9&quot;&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt;9&amp;lt;/span&amp;gt;&amp;lt;/a&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt;Solanki et al&amp;lt;/span&amp;gt;&amp;lt;a href=&quot;https://jamanetwork.com/journals/jama/fullarticle/2588763#joi160132r12&quot;&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt;12&amp;lt;/span&amp;gt;&amp;lt;/a&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt; reported a sensitivity of 93.8% at a specificity of 72.2% on the same data set. A study by Philip et al&amp;lt;/span&amp;gt;&amp;lt;a href=&quot;https://jamanetwork.com/journals/jama/fullarticle/2588763#joi160132r21&quot;&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt;21&amp;lt;/span&amp;gt;&amp;lt;/a&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt; reported a sensitivity of 86.2% at a specificity of 76.8% for predicting disease vs no disease on their own data set of 14, 406 images.’ &amp;lt;/span&amp;gt;&amp;lt;a href=&quot;https://jamanetwork.com/journals/jama/fullarticle/2588763&quot;&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt;https://jamanetwork.com/journals/jama/fullarticle/2588763&amp;lt;/span&amp;gt;&amp;lt;/a&amp;gt;&amp;lt;span style=&quot;font-weight: 400;&quot;&amp;gt; &#039;&gt;&lt;sup&gt;6&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;&lt;/span&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;p&gt;&lt;span style=&quot;font-weight: 400;&quot;&gt;Thus it took about&lt;/span&gt; &lt;b&gt;ten years&lt;/b&gt; &lt;span style=&quot;font-weight: 400;&quot;&gt;to go from just below expert human level performance to slightly superhuman performance.&lt;/span&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + ===== Contributions ===== + + + &lt;HTML&gt; + &lt;p&gt;&lt;em&gt;Aysja Johnson researched and wrote this page. Justis Mills and Katja Grace contributed feedback.&lt;/em&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + ===== Footnotes ===== + + + &lt;HTML&gt; + &lt;ol class=&quot;easy-footnotes-wrapper&quot;&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-1-1241&quot;&gt;&lt;/span&gt; See e.g. &lt;a href=&quot;https://www.nhs.uk/conditions/diabetic-retinopathy/&quot;&gt;https://www.nhs.uk/conditions/diabetic-retinopathy/&lt;/a&gt; &lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-1-1241&quot;&gt;&lt;/a&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-2-1241&quot;&gt;&lt;/span&gt;See &lt;a href=&quot;https://nei.nih.gov/health/diabetic/retinopathy&quot;&gt;https://nei.nih.gov/health/diabetic/retinopathy&lt;/a&gt;&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-2-1241&quot;&gt;&lt;/a&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-3-1241&quot;&gt;&lt;/span&gt; See Results section before adjudication and consensus &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/23494039&quot;&gt;&lt;span style=&quot;font-weight: 400&quot;&gt;https://www.ncbi.nlm.nih.gov/pubmed/23494039&lt;/span&gt;&lt;/a&gt; &lt;span style=&quot;font-weight: 400&quot;&gt;&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-3-1241&quot;&gt;&lt;/a&gt;&lt;/span&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span style=&quot;font-weight: 400&quot;&gt;&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-4-1241&quot;&gt;&lt;/span&gt; See Figure 3&lt;/span&gt; &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2911785/&quot;&gt;&lt;span style=&quot;font-weight: 400&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2911785/&lt;/span&gt;&lt;/a&gt; &lt;span style=&quot;font-weight: 400&quot;&gt;&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-4-1241&quot;&gt;&lt;/a&gt;&lt;/span&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt;&lt;span style=&quot;font-weight: 400&quot;&gt;&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-5-1241&quot;&gt;&lt;/span&gt; &lt;a href=&quot;https://ai.googleblog.com/2016/11/deep-learning-for-detection-of-diabetic.html&quot;&gt;https://ai.googleblog.com/2016/11/deep-learning-for-detection-of-diabetic.html&lt;/a&gt; &lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-5-1241&quot;&gt;&lt;/a&gt;&lt;/span&gt;&lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-6-1241&quot;&gt;&lt;/span&gt; ‘&lt;span style=&quot;font-weight: 400&quot;&gt;Automated and semi-automated diabetic retinopathy evaluation has been previously studied by other groups. Abràmoff et al&lt;/span&gt;&lt;a href=&quot;https://jamanetwork.com/journals/jama/fullarticle/2588763#joi160132r4&quot;&gt;&lt;span style=&quot;font-weight: 400&quot;&gt;4&lt;/span&gt;&lt;/a&gt; &lt;span style=&quot;font-weight: 400&quot;&gt;reported a sensitivity of 96.8% at a specificity of 59.4% for detecting referable diabetic retinopathy on the publicly available Messidor-2 data set.&lt;/span&gt;&lt;a href=&quot;https://jamanetwork.com/journals/jama/fullarticle/2588763#joi160132r9&quot;&gt;&lt;span style=&quot;font-weight: 400&quot;&gt;9&lt;/span&gt;&lt;/a&gt;&lt;span style=&quot;font-weight: 400&quot;&gt;Solanki et al&lt;/span&gt;&lt;a href=&quot;https://jamanetwork.com/journals/jama/fullarticle/2588763#joi160132r12&quot;&gt;&lt;span style=&quot;font-weight: 400&quot;&gt;12&lt;/span&gt;&lt;/a&gt; &lt;span style=&quot;font-weight: 400&quot;&gt;reported a sensitivity of 93.8% at a specificity of 72.2% on the same data set. A study by Philip et al&lt;/span&gt;&lt;a href=&quot;https://jamanetwork.com/journals/jama/fullarticle/2588763#joi160132r21&quot;&gt;&lt;span style=&quot;font-weight: 400&quot;&gt;21&lt;/span&gt;&lt;/a&gt; &lt;span style=&quot;font-weight: 400&quot;&gt;reported a sensitivity of 86.2% at a specificity of 76.8% for predicting disease vs no disease on their own data set of 14, 406 images.’&lt;/span&gt; &lt;a href=&quot;https://jamanetwork.com/journals/jama/fullarticle/2588763&quot;&gt;&lt;span style=&quot;font-weight: 400&quot;&gt;https://jamanetwork.com/journals/jama/fullarticle/2588763&lt;/span&gt;&lt;/a&gt; &lt;span style=&quot;font-weight: 400&quot;&gt;&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-6-1241&quot;&gt;&lt;/a&gt;&lt;/span&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;/ol&gt; + &lt;/HTML&gt; + + </pre>

2022-09-21T07:37:41+00:00 2022-09-21T07:37:41+00:00 https://wiki.aiimpacts.org/speed_of_ai_transition/range_of_human_performance/time_for_ai_to_cross_the_human_performance_range_in_go?rev=1663745861&do=diff Anonymous anonymous@undisclosed.example.com <pre> @@ -1 +1,156 @@ + ====== Time for AI to cross the human performance range in Go ====== + + // Published 15 October, 2020; last updated 08 March, 2021 // + + + + &lt;HTML&gt; + &lt;p&gt;Progress in computer Go performance took:&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;ul&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt;0-19 years to go from the first attempt to playing at human beginner level (&amp;lt;1987)&lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt;&amp;gt;30 years to go from human beginner level to superhuman level (&amp;lt;1987-2017)&lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt;3 years to go from superhuman level to the the current highest performance (2017-2020)&lt;/div&gt;&lt;/li&gt; + &lt;/ul&gt; + &lt;/HTML&gt; + + + + ===== Details ===== + + + ==== Human performance milestones ==== + + + &lt;HTML&gt; + &lt;p&gt;Human go ratings range from 30 kyu (beginner), through 7 dan to at least 9 professional dan.&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-1-2680&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-1-2680&quot; title=&#039;“Go Ranks and Ratings.” In &amp;lt;em&amp;gt;Wikipedia&amp;lt;/em&amp;gt;, June 20, 2020. &amp;lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Go_ranks_and_ratings&amp;amp;amp;oldid=963489455&quot;&amp;gt;https://en.wikipedia.org/w/index.php?title=Go_ranks_and_ratings&amp;amp;amp;oldid=963489455&amp;lt;/a&amp;gt;.&#039;&gt;&lt;sup&gt;1&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; These ratings go downwards through kyu levels, then upward through dan levels, then upward through professional dan levels. The top ratings seem to be &lt;a href=&quot;http://en.wikipedia.org/wiki/Go_ranks_and_ratings#Elo-like_rating_systems_as_used_in_Go&quot;&gt;closer together&lt;/a&gt; than the lower ones, though there are apparently &lt;a href=&quot;https://en.wikipedia.org/wiki/Go_ranks_and_ratings#Winning_probabilities&quot;&gt;multiple systems&lt;/a&gt; which vary)&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-2-2680&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-2-2680&quot; title=&#039;See table: &amp;lt;br&amp;gt;&amp;lt;br&amp;gt;“Go Ranks and Ratings.” In &amp;lt;em&amp;gt;Wikipedia&amp;lt;/em&amp;gt;, June 20, 2020. &amp;lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Go_ranks_and_ratings&amp;amp;amp;oldid=963489455&quot;&amp;gt;https://en.wikipedia.org/w/index.php?title=Go_ranks_and_ratings&amp;amp;amp;oldid=963489455&amp;lt;/a&amp;gt;.&#039;&gt;&lt;sup&gt;2&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + ==== AI achievement of human milestones ==== + + + === Earliest attempt === + + + &lt;HTML&gt; + &lt;p&gt;Wikipedia says the first Go program was written in 1968.&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-3-2680&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-3-2680&quot; title=&#039;&amp;amp;#8220;The first Go program was written by&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Albert_Lindsey_Zobrist&quot;&amp;gt;Albert Lindsey Zobrist&amp;lt;/a&amp;gt;&amp;amp;nbsp;in 1968 as part of his thesis on&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Pattern_recognition&quot;&amp;gt;pattern recognition&amp;lt;/a&amp;gt;.&amp;lt;sup&amp;gt;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_Go#cite_note-11&quot;&amp;gt;[11]&amp;lt;/a&amp;gt;&amp;lt;/sup&amp;gt;&amp;amp;nbsp;It introduced an&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Influence_function_(statistics)&quot;&amp;gt;influence function&amp;lt;/a&amp;gt;&amp;amp;nbsp;to estimate territory and&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Zobrist_hashing&quot;&amp;gt;Zobrist hashing&amp;lt;/a&amp;gt;&amp;amp;nbsp;to detect&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Ko_rule&quot;&amp;gt;ko&amp;lt;/a&amp;gt;.&amp;amp;#8221;&#039;&gt;&lt;sup&gt;3&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; We do not know how well it performed.&lt;/p&gt; + &lt;/HTML&gt; + + + === Beginner level === + + + &lt;HTML&gt; + &lt;p&gt;We have not investigated early Go performance in depth. Figure 1 includes informed guesses about early performance by David Fotland, author of successful Go program, &lt;em&gt;The Many Faces of Go&lt;/em&gt;, and &lt;em&gt;Sensei’s Library&lt;/em&gt;, a Go wiki.&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-4-2680&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-4-2680&quot; title=&quot;&amp;amp;#8220;Figure 25 shows estimates from two&amp;lt;br&amp;gt;sources: David Fotland—author of The Many Faces of Go, an Olympiad-winning Go program—and Sensei’s Library, a collaborative Go wiki. David Fotland warns that the data from before bots played on KGS is poor, as programs tended not to play in human tournaments and so failed to get ratings.&amp;amp;#8221;&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;Grace, Katja. “Algorithmic Progress in Six Domains.” Berkeley, CA: Machine Intelligence Research Institute, 2013.&quot;&gt;&lt;sup&gt;4&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; Fotland says that early data on AI Go performance is poor, since bots did not play in tournaments, so were not rated.&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;figure class=&quot;wp-block-image size-large is-resized&quot;&gt; + &lt;img alt=&quot;&quot; class=&quot;wp-image-372&quot; height=&quot;417&quot; sizes=&quot;(max-width: 576px) 100vw, 576px&quot; src=&quot;https://aiimpacts.org/wp-content/uploads/2014/12/gobothistory-copy-1024x743.jpg&quot; srcset=&quot;https://aiimpacts.org/wp-content/uploads/2014/12/gobothistory-copy-1024x743.jpg 1024w, https://aiimpacts.org/wp-content/uploads/2014/12/gobothistory-copy-300x218.jpg 300w, https://aiimpacts.org/wp-content/uploads/2014/12/gobothistory-copy.jpg 1358w&quot; width=&quot;576&quot;/&gt; + &lt;figcaption&gt; + &lt;strong&gt;Figure 1&lt;/strong&gt;: From &lt;a href=&quot;http://intelligence.org/files/AlgorithmicProgress.pdf&quot;&gt;Grace 2013&lt;/a&gt;. + &lt;/figcaption&gt; + &lt;/figure&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;p&gt;This suggests that by 1987 Go bots were performing better than human beginners. We do not have evidence to pin down the date of human beginner level AI better, but have also not investigated thoroughly (there appears to be more evidence).&lt;/p&gt; + &lt;/HTML&gt; + + + === Superhuman level === + + + &lt;HTML&gt; + &lt;p&gt;In May 2017 AlphaGo beat the top ranked Go player in the world.&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-5-2680&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-5-2680&quot; title=&#039;&amp;amp;#8220;In May 2017, AlphaGo beat&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Ke_Jie&quot;&amp;gt;Ke Jie&amp;lt;/a&amp;gt;, who at the time was ranked top in the world,&amp;lt;sup&amp;gt;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_Go#cite_note-27&quot;&amp;gt;[27]&amp;lt;/a&amp;gt;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_Go#cite_note-28&quot;&amp;gt;[28]&amp;lt;/a&amp;gt;&amp;lt;/sup&amp;gt;&amp;amp;nbsp;in a&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaGo_versus_Ke_Jie&quot;&amp;gt;three-game match&amp;lt;/a&amp;gt;&amp;amp;nbsp;during the&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Future_of_Go_Summit&quot;&amp;gt;Future of Go Summit&amp;lt;/a&amp;gt;.&amp;lt;sup&amp;gt;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_Go#cite_note-wuzhensecond-29&quot;&amp;gt;[29]&amp;lt;/a&amp;gt;&amp;lt;/sup&amp;gt;&amp;amp;#8220;&amp;lt;br&amp;gt;&amp;lt;br&amp;gt;“Computer Go.” In &amp;lt;em&amp;gt;Wikipedia&amp;lt;/em&amp;gt;, July 27, 2020. &amp;lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Computer_Go&amp;amp;amp;oldid=969736537&quot;&amp;gt;https://en.wikipedia.org/w/index.php?title=Computer_Go&amp;amp;amp;oldid=969736537&amp;lt;/a&amp;gt;&#039;&gt;&lt;sup&gt;5&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; This does not imply that AlphaGo was overall better, but a new version in October could beat the May version in 89 games out of 100&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-6-2680&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-6-2680&quot; title=&#039;&amp;amp;#8220;In October 2017, DeepMind revealed a new version of AlphaGo, trained only through self play, that had surpassed all previous versions, beating the Ke Jie version in 89 out of 100 games.&amp;lt;sup&amp;gt;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_Go#cite_note-alphagozero-30&quot;&amp;gt;[30]&amp;lt;/a&amp;gt;&amp;lt;/sup&amp;gt;&amp;amp;#8220;&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;“Computer Go.” In &amp;lt;em&amp;gt;Wikipedia&amp;lt;/em&amp;gt;, July 27, 2020. &amp;lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Computer_Go&amp;amp;amp;oldid=969736537&quot;&amp;gt;https://en.wikipedia.org/w/index.php?title=Computer_Go&amp;amp;amp;oldid=969736537&amp;lt;/a&amp;gt;&#039;&gt;&lt;sup&gt;6&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;, suggesting that if in May it would have beaten Ke Jie in more than 11% of games, the new version would beat Ke Jie more than half the time, i.e. perform better than the best human player. Thus 2017 seems like a reasonable date for top human-level play.&lt;/p&gt; + &lt;/HTML&gt; + + + ==== Times for AI to cross human-relative ranges  ==== + + + &lt;HTML&gt; + &lt;p&gt;Given the above dates, we have:&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;figure class=&quot;wp-block-table&quot;&gt; + &lt;table&gt; + &lt;tbody&gt; + &lt;tr&gt; + &lt;td&gt;Range&lt;/td&gt; + &lt;td&gt;Start&lt;/td&gt; + &lt;td&gt;End&lt;/td&gt; + &lt;td&gt;Duration (years)&lt;/td&gt; + &lt;/tr&gt; + &lt;tr&gt; + &lt;td&gt;First attempt to beginner level&lt;/td&gt; + &lt;td&gt;1968&lt;/td&gt; + &lt;td&gt;&amp;lt;1987&lt;/td&gt; + &lt;td&gt;&amp;lt;19&lt;/td&gt; + &lt;/tr&gt; + &lt;tr&gt; + &lt;td&gt;Beginner to superhuman&lt;/td&gt; + &lt;td&gt;&amp;lt;1987&lt;/td&gt; + &lt;td&gt;2017&lt;/td&gt; + &lt;td&gt;&amp;gt;30&lt;/td&gt; + &lt;/tr&gt; + &lt;tr&gt; + &lt;td&gt;Above superhuman&lt;/td&gt; + &lt;td&gt;2017&lt;/td&gt; + &lt;td&gt;&amp;gt;2020&lt;/td&gt; + &lt;td&gt;&amp;gt;3&lt;/td&gt; + &lt;/tr&gt; + &lt;/tbody&gt; + &lt;/table&gt; + &lt;/figure&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;p&gt;&lt;em&gt;&lt;em&gt;Primary author: Katja Grace&lt;/em&gt;&lt;/em&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + ===== Notes ===== + + + &lt;HTML&gt; + &lt;ol class=&quot;easy-footnotes-wrapper&quot;&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-1-2680&quot;&gt;&lt;/span&gt;“Go Ranks and Ratings.” In &lt;em&gt;Wikipedia&lt;/em&gt;, June 20, 2020. &lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Go_ranks_and_ratings&amp;amp;oldid=963489455&quot;&gt;https://en.wikipedia.org/w/index.php?title=Go_ranks_and_ratings&amp;amp;oldid=963489455&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-1-2680&quot;&gt;&lt;/a&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-2-2680&quot;&gt;&lt;/span&gt;See table:&lt;br/&gt; + &lt;br/&gt; + “Go Ranks and Ratings.” In &lt;em&gt;Wikipedia&lt;/em&gt;, June 20, 2020. &lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Go_ranks_and_ratings&amp;amp;oldid=963489455&quot;&gt;https://en.wikipedia.org/w/index.php?title=Go_ranks_and_ratings&amp;amp;oldid=963489455&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-2-2680&quot;&gt;&lt;/a&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-3-2680&quot;&gt;&lt;/span&gt;“The first Go program was written by &lt;a href=&quot;https://en.wikipedia.org/wiki/Albert_Lindsey_Zobrist&quot;&gt;Albert Lindsey Zobrist&lt;/a&gt; in 1968 as part of his thesis on &lt;a href=&quot;https://en.wikipedia.org/wiki/Pattern_recognition&quot;&gt;pattern recognition&lt;/a&gt;.&lt;sup&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_Go#cite_note-11&quot;&gt;[11]&lt;/a&gt;&lt;/sup&gt; It introduced an &lt;a href=&quot;https://en.wikipedia.org/wiki/Influence_function_(statistics)&quot;&gt;influence function&lt;/a&gt; to estimate territory and &lt;a href=&quot;https://en.wikipedia.org/wiki/Zobrist_hashing&quot;&gt;Zobrist hashing&lt;/a&gt; to detect &lt;a href=&quot;https://en.wikipedia.org/wiki/Ko_rule&quot;&gt;ko&lt;/a&gt;.”&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-3-2680&quot;&gt;&lt;/a&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-4-2680&quot;&gt;&lt;/span&gt;“Figure 25 shows estimates from two&lt;br/&gt; + sources: David Fotland—author of The Many Faces of Go, an Olympiad-winning Go program—and Sensei’s Library, a collaborative Go wiki. David Fotland warns that the data from before bots played on KGS is poor, as programs tended not to play in human tournaments and so failed to get ratings.” + &lt;p&gt;Grace, Katja. “Algorithmic Progress in Six Domains.” Berkeley, CA: Machine Intelligence Research Institute, 2013.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-4-2680&quot;&gt;&lt;/a&gt;&lt;/p&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-5-2680&quot;&gt;&lt;/span&gt;“In May 2017, AlphaGo beat &lt;a href=&quot;https://en.wikipedia.org/wiki/Ke_Jie&quot;&gt;Ke Jie&lt;/a&gt;, who at the time was ranked top in the world,&lt;sup&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_Go#cite_note-27&quot;&gt;[27]&lt;/a&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_Go#cite_note-28&quot;&gt;[28]&lt;/a&gt;&lt;/sup&gt; in a &lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaGo_versus_Ke_Jie&quot;&gt;three-game match&lt;/a&gt; during the &lt;a href=&quot;https://en.wikipedia.org/wiki/Future_of_Go_Summit&quot;&gt;Future of Go Summit&lt;/a&gt;.&lt;sup&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_Go#cite_note-wuzhensecond-29&quot;&gt;[29]&lt;/a&gt;&lt;/sup&gt;“&lt;br/&gt; + &lt;br/&gt; + “Computer Go.” In &lt;em&gt;Wikipedia&lt;/em&gt;, July 27, 2020. &lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Computer_Go&amp;amp;oldid=969736537&quot;&gt;https://en.wikipedia.org/w/index.php?title=Computer_Go&amp;amp;oldid=969736537&lt;/a&gt;&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-5-2680&quot;&gt;&lt;/a&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-6-2680&quot;&gt;&lt;/span&gt;“In October 2017, DeepMind revealed a new version of AlphaGo, trained only through self play, that had surpassed all previous versions, beating the Ke Jie version in 89 out of 100 games.&lt;sup&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_Go#cite_note-alphagozero-30&quot;&gt;[30]&lt;/a&gt;&lt;/sup&gt;“ + &lt;p&gt;“Computer Go.” In &lt;em&gt;Wikipedia&lt;/em&gt;, July 27, 2020. &lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Computer_Go&amp;amp;oldid=969736537&quot;&gt;https://en.wikipedia.org/w/index.php?title=Computer_Go&amp;amp;oldid=969736537&lt;/a&gt;&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-6-2680&quot;&gt;&lt;/a&gt;&lt;/p&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;/ol&gt; + &lt;/HTML&gt; + + </pre>

<pre> @@ -1 +1,156 @@ + ====== Time for AI to cross the human performance range in Go ====== + + // Published 15 October, 2020; last updated 08 March, 2021 // + + + + &lt;HTML&gt; + &lt;p&gt;Progress in computer Go performance took:&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;ul&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt;0-19 years to go from the first attempt to playing at human beginner level (&amp;lt;1987)&lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt;&amp;gt;30 years to go from human beginner level to superhuman level (&amp;lt;1987-2017)&lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt;3 years to go from superhuman level to the the current highest performance (2017-2020)&lt;/div&gt;&lt;/li&gt; + &lt;/ul&gt; + &lt;/HTML&gt; + + + + ===== Details ===== + + + ==== Human performance milestones ==== + + + &lt;HTML&gt; + &lt;p&gt;Human go ratings range from 30 kyu (beginner), through 7 dan to at least 9 professional dan.&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-1-2680&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-1-2680&quot; title=&#039;“Go Ranks and Ratings.” In &amp;lt;em&amp;gt;Wikipedia&amp;lt;/em&amp;gt;, June 20, 2020. &amp;lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Go_ranks_and_ratings&amp;amp;amp;oldid=963489455&quot;&amp;gt;https://en.wikipedia.org/w/index.php?title=Go_ranks_and_ratings&amp;amp;amp;oldid=963489455&amp;lt;/a&amp;gt;.&#039;&gt;&lt;sup&gt;1&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; These ratings go downwards through kyu levels, then upward through dan levels, then upward through professional dan levels. The top ratings seem to be &lt;a href=&quot;http://en.wikipedia.org/wiki/Go_ranks_and_ratings#Elo-like_rating_systems_as_used_in_Go&quot;&gt;closer together&lt;/a&gt; than the lower ones, though there are apparently &lt;a href=&quot;https://en.wikipedia.org/wiki/Go_ranks_and_ratings#Winning_probabilities&quot;&gt;multiple systems&lt;/a&gt; which vary)&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-2-2680&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-2-2680&quot; title=&#039;See table: &amp;lt;br&amp;gt;&amp;lt;br&amp;gt;“Go Ranks and Ratings.” In &amp;lt;em&amp;gt;Wikipedia&amp;lt;/em&amp;gt;, June 20, 2020. &amp;lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Go_ranks_and_ratings&amp;amp;amp;oldid=963489455&quot;&amp;gt;https://en.wikipedia.org/w/index.php?title=Go_ranks_and_ratings&amp;amp;amp;oldid=963489455&amp;lt;/a&amp;gt;.&#039;&gt;&lt;sup&gt;2&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + ==== AI achievement of human milestones ==== + + + === Earliest attempt === + + + &lt;HTML&gt; + &lt;p&gt;Wikipedia says the first Go program was written in 1968.&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-3-2680&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-3-2680&quot; title=&#039;&amp;amp;#8220;The first Go program was written by&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Albert_Lindsey_Zobrist&quot;&amp;gt;Albert Lindsey Zobrist&amp;lt;/a&amp;gt;&amp;amp;nbsp;in 1968 as part of his thesis on&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Pattern_recognition&quot;&amp;gt;pattern recognition&amp;lt;/a&amp;gt;.&amp;lt;sup&amp;gt;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_Go#cite_note-11&quot;&amp;gt;[11]&amp;lt;/a&amp;gt;&amp;lt;/sup&amp;gt;&amp;amp;nbsp;It introduced an&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Influence_function_(statistics)&quot;&amp;gt;influence function&amp;lt;/a&amp;gt;&amp;amp;nbsp;to estimate territory and&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Zobrist_hashing&quot;&amp;gt;Zobrist hashing&amp;lt;/a&amp;gt;&amp;amp;nbsp;to detect&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Ko_rule&quot;&amp;gt;ko&amp;lt;/a&amp;gt;.&amp;amp;#8221;&#039;&gt;&lt;sup&gt;3&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; We do not know how well it performed.&lt;/p&gt; + &lt;/HTML&gt; + + + === Beginner level === + + + &lt;HTML&gt; + &lt;p&gt;We have not investigated early Go performance in depth. Figure 1 includes informed guesses about early performance by David Fotland, author of successful Go program, &lt;em&gt;The Many Faces of Go&lt;/em&gt;, and &lt;em&gt;Sensei’s Library&lt;/em&gt;, a Go wiki.&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-4-2680&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-4-2680&quot; title=&quot;&amp;amp;#8220;Figure 25 shows estimates from two&amp;lt;br&amp;gt;sources: David Fotland—author of The Many Faces of Go, an Olympiad-winning Go program—and Sensei’s Library, a collaborative Go wiki. David Fotland warns that the data from before bots played on KGS is poor, as programs tended not to play in human tournaments and so failed to get ratings.&amp;amp;#8221;&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;Grace, Katja. “Algorithmic Progress in Six Domains.” Berkeley, CA: Machine Intelligence Research Institute, 2013.&quot;&gt;&lt;sup&gt;4&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; Fotland says that early data on AI Go performance is poor, since bots did not play in tournaments, so were not rated.&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;figure class=&quot;wp-block-image size-large is-resized&quot;&gt; + &lt;img alt=&quot;&quot; class=&quot;wp-image-372&quot; height=&quot;417&quot; sizes=&quot;(max-width: 576px) 100vw, 576px&quot; src=&quot;https://aiimpacts.org/wp-content/uploads/2014/12/gobothistory-copy-1024x743.jpg&quot; srcset=&quot;https://aiimpacts.org/wp-content/uploads/2014/12/gobothistory-copy-1024x743.jpg 1024w, https://aiimpacts.org/wp-content/uploads/2014/12/gobothistory-copy-300x218.jpg 300w, https://aiimpacts.org/wp-content/uploads/2014/12/gobothistory-copy.jpg 1358w&quot; width=&quot;576&quot;/&gt; + &lt;figcaption&gt; + &lt;strong&gt;Figure 1&lt;/strong&gt;: From &lt;a href=&quot;http://intelligence.org/files/AlgorithmicProgress.pdf&quot;&gt;Grace 2013&lt;/a&gt;. + &lt;/figcaption&gt; + &lt;/figure&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;p&gt;This suggests that by 1987 Go bots were performing better than human beginners. We do not have evidence to pin down the date of human beginner level AI better, but have also not investigated thoroughly (there appears to be more evidence).&lt;/p&gt; + &lt;/HTML&gt; + + + === Superhuman level === + + + &lt;HTML&gt; + &lt;p&gt;In May 2017 AlphaGo beat the top ranked Go player in the world.&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-5-2680&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-5-2680&quot; title=&#039;&amp;amp;#8220;In May 2017, AlphaGo beat&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Ke_Jie&quot;&amp;gt;Ke Jie&amp;lt;/a&amp;gt;, who at the time was ranked top in the world,&amp;lt;sup&amp;gt;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_Go#cite_note-27&quot;&amp;gt;[27]&amp;lt;/a&amp;gt;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_Go#cite_note-28&quot;&amp;gt;[28]&amp;lt;/a&amp;gt;&amp;lt;/sup&amp;gt;&amp;amp;nbsp;in a&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaGo_versus_Ke_Jie&quot;&amp;gt;three-game match&amp;lt;/a&amp;gt;&amp;amp;nbsp;during the&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Future_of_Go_Summit&quot;&amp;gt;Future of Go Summit&amp;lt;/a&amp;gt;.&amp;lt;sup&amp;gt;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_Go#cite_note-wuzhensecond-29&quot;&amp;gt;[29]&amp;lt;/a&amp;gt;&amp;lt;/sup&amp;gt;&amp;amp;#8220;&amp;lt;br&amp;gt;&amp;lt;br&amp;gt;“Computer Go.” In &amp;lt;em&amp;gt;Wikipedia&amp;lt;/em&amp;gt;, July 27, 2020. &amp;lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Computer_Go&amp;amp;amp;oldid=969736537&quot;&amp;gt;https://en.wikipedia.org/w/index.php?title=Computer_Go&amp;amp;amp;oldid=969736537&amp;lt;/a&amp;gt;&#039;&gt;&lt;sup&gt;5&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; This does not imply that AlphaGo was overall better, but a new version in October could beat the May version in 89 games out of 100&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-6-2680&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-6-2680&quot; title=&#039;&amp;amp;#8220;In October 2017, DeepMind revealed a new version of AlphaGo, trained only through self play, that had surpassed all previous versions, beating the Ke Jie version in 89 out of 100 games.&amp;lt;sup&amp;gt;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_Go#cite_note-alphagozero-30&quot;&amp;gt;[30]&amp;lt;/a&amp;gt;&amp;lt;/sup&amp;gt;&amp;amp;#8220;&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;“Computer Go.” In &amp;lt;em&amp;gt;Wikipedia&amp;lt;/em&amp;gt;, July 27, 2020. &amp;lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Computer_Go&amp;amp;amp;oldid=969736537&quot;&amp;gt;https://en.wikipedia.org/w/index.php?title=Computer_Go&amp;amp;amp;oldid=969736537&amp;lt;/a&amp;gt;&#039;&gt;&lt;sup&gt;6&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;, suggesting that if in May it would have beaten Ke Jie in more than 11% of games, the new version would beat Ke Jie more than half the time, i.e. perform better than the best human player. Thus 2017 seems like a reasonable date for top human-level play.&lt;/p&gt; + &lt;/HTML&gt; + + + ==== Times for AI to cross human-relative ranges  ==== + + + &lt;HTML&gt; + &lt;p&gt;Given the above dates, we have:&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;figure class=&quot;wp-block-table&quot;&gt; + &lt;table&gt; + &lt;tbody&gt; + &lt;tr&gt; + &lt;td&gt;Range&lt;/td&gt; + &lt;td&gt;Start&lt;/td&gt; + &lt;td&gt;End&lt;/td&gt; + &lt;td&gt;Duration (years)&lt;/td&gt; + &lt;/tr&gt; + &lt;tr&gt; + &lt;td&gt;First attempt to beginner level&lt;/td&gt; + &lt;td&gt;1968&lt;/td&gt; + &lt;td&gt;&amp;lt;1987&lt;/td&gt; + &lt;td&gt;&amp;lt;19&lt;/td&gt; + &lt;/tr&gt; + &lt;tr&gt; + &lt;td&gt;Beginner to superhuman&lt;/td&gt; + &lt;td&gt;&amp;lt;1987&lt;/td&gt; + &lt;td&gt;2017&lt;/td&gt; + &lt;td&gt;&amp;gt;30&lt;/td&gt; + &lt;/tr&gt; + &lt;tr&gt; + &lt;td&gt;Above superhuman&lt;/td&gt; + &lt;td&gt;2017&lt;/td&gt; + &lt;td&gt;&amp;gt;2020&lt;/td&gt; + &lt;td&gt;&amp;gt;3&lt;/td&gt; + &lt;/tr&gt; + &lt;/tbody&gt; + &lt;/table&gt; + &lt;/figure&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;p&gt;&lt;em&gt;&lt;em&gt;Primary author: Katja Grace&lt;/em&gt;&lt;/em&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + ===== Notes ===== + + + &lt;HTML&gt; + &lt;ol class=&quot;easy-footnotes-wrapper&quot;&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-1-2680&quot;&gt;&lt;/span&gt;“Go Ranks and Ratings.” In &lt;em&gt;Wikipedia&lt;/em&gt;, June 20, 2020. &lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Go_ranks_and_ratings&amp;amp;oldid=963489455&quot;&gt;https://en.wikipedia.org/w/index.php?title=Go_ranks_and_ratings&amp;amp;oldid=963489455&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-1-2680&quot;&gt;&lt;/a&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-2-2680&quot;&gt;&lt;/span&gt;See table:&lt;br/&gt; + &lt;br/&gt; + “Go Ranks and Ratings.” In &lt;em&gt;Wikipedia&lt;/em&gt;, June 20, 2020. &lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Go_ranks_and_ratings&amp;amp;oldid=963489455&quot;&gt;https://en.wikipedia.org/w/index.php?title=Go_ranks_and_ratings&amp;amp;oldid=963489455&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-2-2680&quot;&gt;&lt;/a&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-3-2680&quot;&gt;&lt;/span&gt;“The first Go program was written by &lt;a href=&quot;https://en.wikipedia.org/wiki/Albert_Lindsey_Zobrist&quot;&gt;Albert Lindsey Zobrist&lt;/a&gt; in 1968 as part of his thesis on &lt;a href=&quot;https://en.wikipedia.org/wiki/Pattern_recognition&quot;&gt;pattern recognition&lt;/a&gt;.&lt;sup&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_Go#cite_note-11&quot;&gt;[11]&lt;/a&gt;&lt;/sup&gt; It introduced an &lt;a href=&quot;https://en.wikipedia.org/wiki/Influence_function_(statistics)&quot;&gt;influence function&lt;/a&gt; to estimate territory and &lt;a href=&quot;https://en.wikipedia.org/wiki/Zobrist_hashing&quot;&gt;Zobrist hashing&lt;/a&gt; to detect &lt;a href=&quot;https://en.wikipedia.org/wiki/Ko_rule&quot;&gt;ko&lt;/a&gt;.”&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-3-2680&quot;&gt;&lt;/a&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-4-2680&quot;&gt;&lt;/span&gt;“Figure 25 shows estimates from two&lt;br/&gt; + sources: David Fotland—author of The Many Faces of Go, an Olympiad-winning Go program—and Sensei’s Library, a collaborative Go wiki. David Fotland warns that the data from before bots played on KGS is poor, as programs tended not to play in human tournaments and so failed to get ratings.” + &lt;p&gt;Grace, Katja. “Algorithmic Progress in Six Domains.” Berkeley, CA: Machine Intelligence Research Institute, 2013.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-4-2680&quot;&gt;&lt;/a&gt;&lt;/p&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-5-2680&quot;&gt;&lt;/span&gt;“In May 2017, AlphaGo beat &lt;a href=&quot;https://en.wikipedia.org/wiki/Ke_Jie&quot;&gt;Ke Jie&lt;/a&gt;, who at the time was ranked top in the world,&lt;sup&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_Go#cite_note-27&quot;&gt;[27]&lt;/a&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_Go#cite_note-28&quot;&gt;[28]&lt;/a&gt;&lt;/sup&gt; in a &lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaGo_versus_Ke_Jie&quot;&gt;three-game match&lt;/a&gt; during the &lt;a href=&quot;https://en.wikipedia.org/wiki/Future_of_Go_Summit&quot;&gt;Future of Go Summit&lt;/a&gt;.&lt;sup&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_Go#cite_note-wuzhensecond-29&quot;&gt;[29]&lt;/a&gt;&lt;/sup&gt;“&lt;br/&gt; + &lt;br/&gt; + “Computer Go.” In &lt;em&gt;Wikipedia&lt;/em&gt;, July 27, 2020. &lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Computer_Go&amp;amp;oldid=969736537&quot;&gt;https://en.wikipedia.org/w/index.php?title=Computer_Go&amp;amp;oldid=969736537&lt;/a&gt;&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-5-2680&quot;&gt;&lt;/a&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-6-2680&quot;&gt;&lt;/span&gt;“In October 2017, DeepMind revealed a new version of AlphaGo, trained only through self play, that had surpassed all previous versions, beating the Ke Jie version in 89 out of 100 games.&lt;sup&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_Go#cite_note-alphagozero-30&quot;&gt;[30]&lt;/a&gt;&lt;/sup&gt;“ + &lt;p&gt;“Computer Go.” In &lt;em&gt;Wikipedia&lt;/em&gt;, July 27, 2020. &lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Computer_Go&amp;amp;oldid=969736537&quot;&gt;https://en.wikipedia.org/w/index.php?title=Computer_Go&amp;amp;oldid=969736537&lt;/a&gt;&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-6-2680&quot;&gt;&lt;/a&gt;&lt;/p&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;/ol&gt; + &lt;/HTML&gt; + + </pre>

2022-09-21T07:37:41+00:00 2022-09-21T07:37:41+00:00 https://wiki.aiimpacts.org/speed_of_ai_transition/range_of_human_performance/time_for_ai_to_cross_the_human_range_in_english_draughts?rev=1663745861&do=diff Anonymous anonymous@undisclosed.example.com <pre> @@ -1 +1,204 @@ + ====== Time for AI to cross the human range in English draughts ====== + + // Published 26 October, 2020; last updated 08 March, 2021 // + + + + &lt;HTML&gt; + &lt;p&gt;AI progress in English draughts performance crossed the following ranges in the following times:&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;figure class=&quot;wp-block-table&quot;&gt; + &lt;table&gt; + &lt;tbody&gt; + &lt;tr&gt; + &lt;td&gt;&lt;strong&gt;Range&lt;/strong&gt;&lt;/td&gt; + &lt;td&gt;&lt;strong&gt;Start&lt;/strong&gt;&lt;/td&gt; + &lt;td&gt;&lt;strong&gt;End&lt;/strong&gt;&lt;/td&gt; + &lt;td&gt;&lt;strong&gt;Duration (years)&lt;/strong&gt;&lt;/td&gt; + &lt;/tr&gt; + &lt;tr&gt; + &lt;td&gt;First attempt to beginner level&lt;/td&gt; + &lt;td&gt;1951&lt;/td&gt; + &lt;td&gt;~1956, &amp;lt;1961&lt;/td&gt; + &lt;td&gt;~4, &amp;lt;10&lt;/td&gt; + &lt;/tr&gt; + &lt;tr&gt; + &lt;td&gt;Beginner to superhuman&lt;/td&gt; + &lt;td&gt;~1956, &amp;lt;1961&lt;/td&gt; + &lt;td&gt;1994&lt;/td&gt; + &lt;td&gt;~38, &amp;gt;33&lt;/td&gt; + &lt;/tr&gt; + &lt;tr&gt; + &lt;td&gt;Above superhuman&lt;/td&gt; + &lt;td&gt;1994&lt;/td&gt; + &lt;td&gt;2007*&lt;/td&gt; + &lt;td&gt;13*&lt;/td&gt; + &lt;/tr&gt; + &lt;/tbody&gt; + &lt;/table&gt; + &lt;figcaption&gt; + * treating perfect play as the end of progress, though progress could potentially be made in performing better against imperfect play. + &lt;/figcaption&gt; + &lt;/figure&gt; + &lt;/HTML&gt; + + + + ===== Details ===== + + + ==== Metric ==== + + + &lt;HTML&gt; + &lt;p&gt;‘English Draughts’ is a popular variety of draughts, or checkers.&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;p&gt;Here we look at direct success of AI in beating human players, rather than measuring humans and AI on a separate metric of strength.&lt;/p&gt; + &lt;/HTML&gt; + + + ==== Data ==== + + + &lt;HTML&gt; + &lt;p&gt;Data here comes mostly from Wikipedia. We found several discrepancies in Wikipedia’s accounts of these events, so consider the remaining data to be somewhat unreliable.&lt;/p&gt; + &lt;/HTML&gt; + + + ==== AI achievement of human milestones ==== + + + === Earliest attempt === + + + &lt;HTML&gt; + &lt;p&gt;According to Wikipedia, the first checkers program was run in 1951.&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-1-2704&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-1-2704&quot; title=&#039;&amp;amp;#8220;The first English draughts&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_program&quot;&amp;gt;computer program&amp;lt;/a&amp;gt;&amp;amp;nbsp;was written by&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Christopher_Strachey&quot;&amp;gt;Christopher Strachey&amp;lt;/a&amp;gt;, M.A. at the&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/National_Physical_Laboratory_(United_Kingdom)&quot;&amp;gt;National Physical Laboratory&amp;lt;/a&amp;gt;, London.&amp;lt;sup&amp;gt;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/English_draughts#cite_note-5&quot;&amp;gt;[5]&amp;lt;/a&amp;gt;&amp;lt;/sup&amp;gt;&amp;amp;nbsp;Strachey finished the programme, written in his spare time, in February 1951. It ran for the first time on NPL&amp;amp;#8217;s&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Pilot_ACE&quot;&amp;gt;Pilot ACE&amp;lt;/a&amp;gt;&amp;amp;nbsp;on 30 July 1951. He soon modified the programme to run on the&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Manchester_Mark_1&quot;&amp;gt;Manchester Mark 1&amp;lt;/a&amp;gt;.&amp;amp;#8221;&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;“English Draughts.” In &amp;lt;em&amp;gt;Wikipedia&amp;lt;/em&amp;gt;, October 8, 2020. &amp;lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=English_draughts&amp;amp;amp;oldid=982525532&quot;&amp;gt;https://en.wikipedia.org/w/index.php?title=English_draughts&amp;amp;amp;oldid=982525532&amp;lt;/a&amp;gt;.&#039;&gt;&lt;sup&gt;1&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + === Beginner level === + + + &lt;HTML&gt; + &lt;p&gt;There seems to be some ambiguity around the timing and performance of Arthur Samuel’s early draughts programs, but it appears that he worked on them from around 1952. In 1956, he demonstrated a program on television. &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-2-2704&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-2-2704&quot; title=&#039;&amp;amp;#8220;It didn’t take long before Samuel had a program that played a respectable game of checkers, capable of easily defeating novice players. It was first publicly demonstrated on television on February 24, 1956. Thomas Watson, President of IBM, arranged for the program to be exhibited to shareholders. He predicted that it would result in a fifteen-point rise in the IBM stock price. It did.&amp;amp;#8221;&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;Schaeffer, Jonathan. &amp;lt;em&amp;gt;One Jump Ahead: Challenging Human Supremacy in Checkers&amp;lt;/em&amp;gt;. 1st Edition. New York: Springer, 1997.&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;quoted in:&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;“Legacy &amp;amp;#8211; Chinook &amp;amp;#8211; World Man-Machine Checkers Champion.” Accessed October 26, 2020. &amp;lt;a href=&quot;http://webdocs.cs.ualberta.ca/~chinook/project/legacy.html&quot;&amp;gt;http://webdocs.cs.ualberta.ca/~chinook/project/legacy.html&amp;lt;/a&amp;gt;.&#039;&gt;&lt;sup&gt;2&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; It is unclear how good the program’s play was, but it is said to have resulted in a fifteen-point rise in the stock price of IBM when demonstrated to IBM shareholders, seemingly at around that time. This weakly suggests that the program played at at least beginner level.&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;p&gt;In 1962 Samuel’s program apparently beat an ambiguously skilled player who would by four-years later become a state champion in Connecticut.&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-3-2704&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-3-2704&quot; title=&#039;The historical record is fuzzy, but for some reason he chose to have his program debut against Robert Nealey a blind checkers champion from Stamford, Connecticut. IBM’s&amp;lt;em&amp;gt;&amp;amp;nbsp;Research News&amp;lt;/em&amp;gt;&amp;amp;nbsp;claimed that Nealey was “a former Connecticut checkers champion, and one of the nation’s foremost players.”&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;Although a self-proclaimed master, Nealey’s tournament results never justified such a claim. At the time of the game, he was not a former Connecticut state champion, although he did win the title in 1966, four years after the game with Samuel’s program. In the history of the state champi­onship, no recognized master had won the event. Nealey didn’t play in any of the premier checkers events, such as the U.S. championship, and apparently acquired his reputation by beating local players.&amp;amp;#8221;&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;Schaeffer, Jonathan. &amp;lt;em&amp;gt;One Jump Ahead: Challenging Human Supremacy in Checkers&amp;lt;/em&amp;gt;. 1st Edition. New York: Springer, 1997.&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;quoted in: &amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;“Legacy &amp;amp;#8211; Chinook &amp;amp;#8211; World Man-Machine Checkers Champion.” Accessed October 26, 2020. &amp;lt;a href=&quot;http://webdocs.cs.ualberta.ca/~chinook/project/legacy.html&quot;&amp;gt;http://webdocs.cs.ualberta.ca/~chinook/project/legacy.html&amp;lt;/a&amp;gt;.&#039;&gt;&lt;sup&gt;3&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; Thus progress was definitely beyond beginner level by 1962.&lt;/p&gt; + &lt;/HTML&gt; + + + === Superhuman level === + + + &lt;HTML&gt; + &lt;p&gt;In 1994, computer program Chinook drew six times against world champion Marius Tinsley, before Tinsley withdrew due to pancreatic cancer and Chinook officially won.&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-4-2704&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-4-2704&quot; title=&#039;&amp;amp;#8220;In a rematch, Chinook was declared the Man-Machine World Champion in checkers in 1994 in a match against Marion Tinsley after six drawn games, and Tinsley&amp;amp;#8217;s withdrawal due to pancreatic cancer. While Chinook became the world champion, it never defeated the best checkers player of all time, Tinsley, who was significantly superior to even his closest peer.&amp;lt;sup&amp;gt;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Chinook_(computer_program)#cite_note-1&quot;&amp;gt;[1]&amp;lt;/a&amp;gt;&amp;lt;/sup&amp;gt;&amp;amp;#8220;&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;“Chinook (Computer Program).” In &amp;lt;em&amp;gt;Wikipedia&amp;lt;/em&amp;gt;, August 24, 2020. &amp;lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Chinook_(computer_program)&amp;amp;amp;oldid=974711515&quot;&amp;gt;https://en.wikipedia.org/w/index.php?title=Chinook_(computer_program)&amp;amp;amp;oldid=974711515&amp;lt;/a&amp;gt;.&#039;&gt;&lt;sup&gt;4&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; Thus Chinook appears to have been close to as good as the best player in 1994.&lt;/p&gt; + &lt;/HTML&gt; + + + === End of progress === + + + &lt;HTML&gt; + &lt;p&gt;In 2007 checkers was ‘weakly solved’, which is to say that perfect play guaranteeing a draw for both sides from the start of the game is known, from the starting state&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-5-2704&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-5-2704&quot; title=&#039;&amp;amp;#8220;English draughts was&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game&quot;&amp;gt;weakly solved&amp;lt;/a&amp;gt;&amp;amp;nbsp;in 2007 by the team of Canadian computer scientist&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Jonathan_Schaeffer&quot;&amp;gt;Jonathan Schaeffer&amp;lt;/a&amp;gt;. From the standard starting position, both players can guarantee a draw with&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Perfect_play&quot;&amp;gt;perfect play&amp;lt;/a&amp;gt;.&amp;amp;#8221;&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;“Draughts.” In &amp;lt;em&amp;gt;Wikipedia&amp;lt;/em&amp;gt;, October 25, 2020. &amp;lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Draughts&amp;amp;amp;oldid=985336948&quot;&amp;gt;https://en.wikipedia.org/w/index.php?title=Draughts&amp;amp;amp;oldid=985336948&amp;lt;/a&amp;gt;.&#039;&gt;&lt;sup&gt;5&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; (this does not imply that if someone plays imperfectly, perfect moves following this are known).&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-6-2704&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-6-2704&quot; title=&#039;&amp;amp;#8220;A&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Two-player_game&quot;&amp;gt;two-player game&amp;lt;/a&amp;gt;&amp;amp;nbsp;can be solved on several levels:&amp;lt;sup&amp;gt;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game#cite_note-Allis-1&quot;&amp;gt;[1]&amp;lt;/a&amp;gt;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game#cite_note-2&quot;&amp;gt;[2]&amp;lt;/a&amp;gt;&amp;lt;/sup&amp;gt;&amp;amp;#8230;&amp;lt;br&amp;gt;&amp;lt;strong&amp;gt;Weak&amp;lt;/strong&amp;gt; &amp;lt;br&amp;gt;Provide an algorithm that secures a win for one player, or a draw for either, against any possible moves by the opponent, from the beginning of the game. That is, produce at least one complete ideal game (all moves start to end) with proof that each move is optimal for the player making it. It does not necessarily mean a computer program using the solution will play optimally against an imperfect opponent.&amp;amp;#8221;&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;“Solved Game.” In &amp;lt;em&amp;gt;Wikipedia&amp;lt;/em&amp;gt;, September 11, 2020. &amp;lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Solved_game&amp;amp;amp;oldid=977794097&quot;&amp;gt;https://en.wikipedia.org/w/index.php?title=Solved_game&amp;amp;amp;oldid=977794097&amp;lt;/a&amp;gt;.&#039;&gt;&lt;sup&gt;6&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; This is not the best possible performance by all measures, since further progress could presumably be made on reliably beating worse players.&lt;/p&gt; + &lt;/HTML&gt; + + + ==== Times for AI to cross human-relative ranges  ==== + + + &lt;HTML&gt; + &lt;p&gt;Given the above dates, we have:&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;figure class=&quot;wp-block-table&quot;&gt; + &lt;table&gt; + &lt;tbody&gt; + &lt;tr&gt; + &lt;td&gt;&lt;strong&gt;Range&lt;/strong&gt;&lt;/td&gt; + &lt;td&gt;&lt;strong&gt;Start&lt;/strong&gt;&lt;/td&gt; + &lt;td&gt;&lt;strong&gt;End&lt;/strong&gt;&lt;/td&gt; + &lt;td&gt;&lt;strong&gt;Duration (years)&lt;/strong&gt;&lt;/td&gt; + &lt;/tr&gt; + &lt;tr&gt; + &lt;td&gt;First attempt to beginner level&lt;/td&gt; + &lt;td&gt;1951&lt;/td&gt; + &lt;td&gt;~1956, &amp;lt;1961&lt;/td&gt; + &lt;td&gt;~4, &amp;lt;10&lt;/td&gt; + &lt;/tr&gt; + &lt;tr&gt; + &lt;td&gt;Beginner to superhuman&lt;/td&gt; + &lt;td&gt;~1956, &amp;lt;1961&lt;/td&gt; + &lt;td&gt;1994&lt;/td&gt; + &lt;td&gt;~38, &amp;gt;33&lt;/td&gt; + &lt;/tr&gt; + &lt;tr&gt; + &lt;td&gt;Above superhuman&lt;/td&gt; + &lt;td&gt;1994&lt;/td&gt; + &lt;td&gt;2007*&lt;/td&gt; + &lt;td&gt;13*&lt;/td&gt; + &lt;/tr&gt; + &lt;/tbody&gt; + &lt;/table&gt; + &lt;figcaption&gt; + * treating perfect play as the end of progress, though progress could potentially be made in performing better against imperfect play. + &lt;/figcaption&gt; + &lt;/figure&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;p&gt;&lt;em&gt;&lt;em&gt;Primary author: Katja Grace&lt;/em&gt;&lt;/em&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + ===== Notes ===== + + + &lt;HTML&gt; + &lt;ol class=&quot;easy-footnotes-wrapper&quot;&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-1-2704&quot;&gt;&lt;/span&gt;“The first English draughts &lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_program&quot;&gt;computer program&lt;/a&gt; was written by &lt;a href=&quot;https://en.wikipedia.org/wiki/Christopher_Strachey&quot;&gt;Christopher Strachey&lt;/a&gt;, M.A. at the &lt;a href=&quot;https://en.wikipedia.org/wiki/National_Physical_Laboratory_(United_Kingdom)&quot;&gt;National Physical Laboratory&lt;/a&gt;, London.&lt;sup&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/English_draughts#cite_note-5&quot;&gt;[5]&lt;/a&gt;&lt;/sup&gt; Strachey finished the programme, written in his spare time, in February 1951. It ran for the first time on NPL’s &lt;a href=&quot;https://en.wikipedia.org/wiki/Pilot_ACE&quot;&gt;Pilot ACE&lt;/a&gt; on 30 July 1951. He soon modified the programme to run on the &lt;a href=&quot;https://en.wikipedia.org/wiki/Manchester_Mark_1&quot;&gt;Manchester Mark 1&lt;/a&gt;.” + &lt;p&gt;“English Draughts.” In &lt;em&gt;Wikipedia&lt;/em&gt;, October 8, 2020. &lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=English_draughts&amp;amp;oldid=982525532&quot;&gt;https://en.wikipedia.org/w/index.php?title=English_draughts&amp;amp;oldid=982525532&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-1-2704&quot;&gt;&lt;/a&gt;&lt;/p&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-2-2704&quot;&gt;&lt;/span&gt;“It didn’t take long before Samuel had a program that played a respectable game of checkers, capable of easily defeating novice players. It was first publicly demonstrated on television on February 24, 1956. Thomas Watson, President of IBM, arranged for the program to be exhibited to shareholders. He predicted that it would result in a fifteen-point rise in the IBM stock price. It did.” + &lt;p&gt;Schaeffer, Jonathan. &lt;em&gt;One Jump Ahead: Challenging Human Supremacy in Checkers&lt;/em&gt;. 1st Edition. New York: Springer, 1997.&lt;/p&gt; + &lt;p&gt;quoted in:&lt;/p&gt; + &lt;p&gt;“Legacy – Chinook – World Man-Machine Checkers Champion.” Accessed October 26, 2020. &lt;a href=&quot;http://webdocs.cs.ualberta.ca/~chinook/project/legacy.html&quot;&gt;http://webdocs.cs.ualberta.ca/~chinook/project/legacy.html&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-2-2704&quot;&gt;&lt;/a&gt;&lt;/p&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-3-2704&quot;&gt;&lt;/span&gt;The historical record is fuzzy, but for some reason he chose to have his program debut against Robert Nealey a blind checkers champion from Stamford, Connecticut. IBM’s&lt;em&gt; Research News&lt;/em&gt; claimed that Nealey was “a former Connecticut checkers champion, and one of the nation’s foremost players.” + &lt;p&gt;Although a self-proclaimed master, Nealey’s tournament results never justified such a claim. At the time of the game, he was not a former Connecticut state champion, although he did win the title in 1966, four years after the game with Samuel’s program. In the history of the state champi­onship, no recognized master had won the event. Nealey didn’t play in any of the premier checkers events, such as the U.S. championship, and apparently acquired his reputation by beating local players.”&lt;/p&gt; + &lt;p&gt;Schaeffer, Jonathan. &lt;em&gt;One Jump Ahead: Challenging Human Supremacy in Checkers&lt;/em&gt;. 1st Edition. New York: Springer, 1997.&lt;/p&gt; + &lt;p&gt;quoted in:&lt;/p&gt; + &lt;p&gt;“Legacy – Chinook – World Man-Machine Checkers Champion.” Accessed October 26, 2020. &lt;a href=&quot;http://webdocs.cs.ualberta.ca/~chinook/project/legacy.html&quot;&gt;http://webdocs.cs.ualberta.ca/~chinook/project/legacy.html&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-3-2704&quot;&gt;&lt;/a&gt;&lt;/p&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-4-2704&quot;&gt;&lt;/span&gt;“In a rematch, Chinook was declared the Man-Machine World Champion in checkers in 1994 in a match against Marion Tinsley after six drawn games, and Tinsley’s withdrawal due to pancreatic cancer. While Chinook became the world champion, it never defeated the best checkers player of all time, Tinsley, who was significantly superior to even his closest peer.&lt;sup&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Chinook_(computer_program)#cite_note-1&quot;&gt;[1]&lt;/a&gt;&lt;/sup&gt;“ + &lt;p&gt;“Chinook (Computer Program).” In &lt;em&gt;Wikipedia&lt;/em&gt;, August 24, 2020. &lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Chinook_(computer_program)&amp;amp;oldid=974711515&quot;&gt;https://en.wikipedia.org/w/index.php?title=Chinook_(computer_program)&amp;amp;oldid=974711515&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-4-2704&quot;&gt;&lt;/a&gt;&lt;/p&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-5-2704&quot;&gt;&lt;/span&gt;“English draughts was &lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game&quot;&gt;weakly solved&lt;/a&gt; in 2007 by the team of Canadian computer scientist &lt;a href=&quot;https://en.wikipedia.org/wiki/Jonathan_Schaeffer&quot;&gt;Jonathan Schaeffer&lt;/a&gt;. From the standard starting position, both players can guarantee a draw with &lt;a href=&quot;https://en.wikipedia.org/wiki/Perfect_play&quot;&gt;perfect play&lt;/a&gt;.” + &lt;p&gt;“Draughts.” In &lt;em&gt;Wikipedia&lt;/em&gt;, October 25, 2020. &lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Draughts&amp;amp;oldid=985336948&quot;&gt;https://en.wikipedia.org/w/index.php?title=Draughts&amp;amp;oldid=985336948&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-5-2704&quot;&gt;&lt;/a&gt;&lt;/p&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-6-2704&quot;&gt;&lt;/span&gt;“A &lt;a href=&quot;https://en.wikipedia.org/wiki/Two-player_game&quot;&gt;two-player game&lt;/a&gt; can be solved on several levels:&lt;sup&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game#cite_note-Allis-1&quot;&gt;[1]&lt;/a&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game#cite_note-2&quot;&gt;[2]&lt;/a&gt;&lt;/sup&gt;…&lt;br/&gt; + &lt;strong&gt;Weak&lt;/strong&gt;&lt;br/&gt; + Provide an algorithm that secures a win for one player, or a draw for either, against any possible moves by the opponent, from the beginning of the game. That is, produce at least one complete ideal game (all moves start to end) with proof that each move is optimal for the player making it. It does not necessarily mean a computer program using the solution will play optimally against an imperfect opponent.” + &lt;p&gt;“Solved Game.” In &lt;em&gt;Wikipedia&lt;/em&gt;, September 11, 2020. &lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Solved_game&amp;amp;oldid=977794097&quot;&gt;https://en.wikipedia.org/w/index.php?title=Solved_game&amp;amp;oldid=977794097&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-6-2704&quot;&gt;&lt;/a&gt;&lt;/p&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;/ol&gt; + &lt;/HTML&gt; + + </pre>

<pre> @@ -1 +1,204 @@ + ====== Time for AI to cross the human range in English draughts ====== + + // Published 26 October, 2020; last updated 08 March, 2021 // + + + + &lt;HTML&gt; + &lt;p&gt;AI progress in English draughts performance crossed the following ranges in the following times:&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;figure class=&quot;wp-block-table&quot;&gt; + &lt;table&gt; + &lt;tbody&gt; + &lt;tr&gt; + &lt;td&gt;&lt;strong&gt;Range&lt;/strong&gt;&lt;/td&gt; + &lt;td&gt;&lt;strong&gt;Start&lt;/strong&gt;&lt;/td&gt; + &lt;td&gt;&lt;strong&gt;End&lt;/strong&gt;&lt;/td&gt; + &lt;td&gt;&lt;strong&gt;Duration (years)&lt;/strong&gt;&lt;/td&gt; + &lt;/tr&gt; + &lt;tr&gt; + &lt;td&gt;First attempt to beginner level&lt;/td&gt; + &lt;td&gt;1951&lt;/td&gt; + &lt;td&gt;~1956, &amp;lt;1961&lt;/td&gt; + &lt;td&gt;~4, &amp;lt;10&lt;/td&gt; + &lt;/tr&gt; + &lt;tr&gt; + &lt;td&gt;Beginner to superhuman&lt;/td&gt; + &lt;td&gt;~1956, &amp;lt;1961&lt;/td&gt; + &lt;td&gt;1994&lt;/td&gt; + &lt;td&gt;~38, &amp;gt;33&lt;/td&gt; + &lt;/tr&gt; + &lt;tr&gt; + &lt;td&gt;Above superhuman&lt;/td&gt; + &lt;td&gt;1994&lt;/td&gt; + &lt;td&gt;2007*&lt;/td&gt; + &lt;td&gt;13*&lt;/td&gt; + &lt;/tr&gt; + &lt;/tbody&gt; + &lt;/table&gt; + &lt;figcaption&gt; + * treating perfect play as the end of progress, though progress could potentially be made in performing better against imperfect play. + &lt;/figcaption&gt; + &lt;/figure&gt; + &lt;/HTML&gt; + + + + ===== Details ===== + + + ==== Metric ==== + + + &lt;HTML&gt; + &lt;p&gt;‘English Draughts’ is a popular variety of draughts, or checkers.&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;p&gt;Here we look at direct success of AI in beating human players, rather than measuring humans and AI on a separate metric of strength.&lt;/p&gt; + &lt;/HTML&gt; + + + ==== Data ==== + + + &lt;HTML&gt; + &lt;p&gt;Data here comes mostly from Wikipedia. We found several discrepancies in Wikipedia’s accounts of these events, so consider the remaining data to be somewhat unreliable.&lt;/p&gt; + &lt;/HTML&gt; + + + ==== AI achievement of human milestones ==== + + + === Earliest attempt === + + + &lt;HTML&gt; + &lt;p&gt;According to Wikipedia, the first checkers program was run in 1951.&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-1-2704&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-1-2704&quot; title=&#039;&amp;amp;#8220;The first English draughts&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_program&quot;&amp;gt;computer program&amp;lt;/a&amp;gt;&amp;amp;nbsp;was written by&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Christopher_Strachey&quot;&amp;gt;Christopher Strachey&amp;lt;/a&amp;gt;, M.A. at the&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/National_Physical_Laboratory_(United_Kingdom)&quot;&amp;gt;National Physical Laboratory&amp;lt;/a&amp;gt;, London.&amp;lt;sup&amp;gt;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/English_draughts#cite_note-5&quot;&amp;gt;[5]&amp;lt;/a&amp;gt;&amp;lt;/sup&amp;gt;&amp;amp;nbsp;Strachey finished the programme, written in his spare time, in February 1951. It ran for the first time on NPL&amp;amp;#8217;s&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Pilot_ACE&quot;&amp;gt;Pilot ACE&amp;lt;/a&amp;gt;&amp;amp;nbsp;on 30 July 1951. He soon modified the programme to run on the&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Manchester_Mark_1&quot;&amp;gt;Manchester Mark 1&amp;lt;/a&amp;gt;.&amp;amp;#8221;&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;“English Draughts.” In &amp;lt;em&amp;gt;Wikipedia&amp;lt;/em&amp;gt;, October 8, 2020. &amp;lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=English_draughts&amp;amp;amp;oldid=982525532&quot;&amp;gt;https://en.wikipedia.org/w/index.php?title=English_draughts&amp;amp;amp;oldid=982525532&amp;lt;/a&amp;gt;.&#039;&gt;&lt;sup&gt;1&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + === Beginner level === + + + &lt;HTML&gt; + &lt;p&gt;There seems to be some ambiguity around the timing and performance of Arthur Samuel’s early draughts programs, but it appears that he worked on them from around 1952. In 1956, he demonstrated a program on television. &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-2-2704&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-2-2704&quot; title=&#039;&amp;amp;#8220;It didn’t take long before Samuel had a program that played a respectable game of checkers, capable of easily defeating novice players. It was first publicly demonstrated on television on February 24, 1956. Thomas Watson, President of IBM, arranged for the program to be exhibited to shareholders. He predicted that it would result in a fifteen-point rise in the IBM stock price. It did.&amp;amp;#8221;&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;Schaeffer, Jonathan. &amp;lt;em&amp;gt;One Jump Ahead: Challenging Human Supremacy in Checkers&amp;lt;/em&amp;gt;. 1st Edition. New York: Springer, 1997.&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;quoted in:&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;“Legacy &amp;amp;#8211; Chinook &amp;amp;#8211; World Man-Machine Checkers Champion.” Accessed October 26, 2020. &amp;lt;a href=&quot;http://webdocs.cs.ualberta.ca/~chinook/project/legacy.html&quot;&amp;gt;http://webdocs.cs.ualberta.ca/~chinook/project/legacy.html&amp;lt;/a&amp;gt;.&#039;&gt;&lt;sup&gt;2&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; It is unclear how good the program’s play was, but it is said to have resulted in a fifteen-point rise in the stock price of IBM when demonstrated to IBM shareholders, seemingly at around that time. This weakly suggests that the program played at at least beginner level.&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;p&gt;In 1962 Samuel’s program apparently beat an ambiguously skilled player who would by four-years later become a state champion in Connecticut.&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-3-2704&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-3-2704&quot; title=&#039;The historical record is fuzzy, but for some reason he chose to have his program debut against Robert Nealey a blind checkers champion from Stamford, Connecticut. IBM’s&amp;lt;em&amp;gt;&amp;amp;nbsp;Research News&amp;lt;/em&amp;gt;&amp;amp;nbsp;claimed that Nealey was “a former Connecticut checkers champion, and one of the nation’s foremost players.”&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;Although a self-proclaimed master, Nealey’s tournament results never justified such a claim. At the time of the game, he was not a former Connecticut state champion, although he did win the title in 1966, four years after the game with Samuel’s program. In the history of the state champi­onship, no recognized master had won the event. Nealey didn’t play in any of the premier checkers events, such as the U.S. championship, and apparently acquired his reputation by beating local players.&amp;amp;#8221;&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;Schaeffer, Jonathan. &amp;lt;em&amp;gt;One Jump Ahead: Challenging Human Supremacy in Checkers&amp;lt;/em&amp;gt;. 1st Edition. New York: Springer, 1997.&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;quoted in: &amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;“Legacy &amp;amp;#8211; Chinook &amp;amp;#8211; World Man-Machine Checkers Champion.” Accessed October 26, 2020. &amp;lt;a href=&quot;http://webdocs.cs.ualberta.ca/~chinook/project/legacy.html&quot;&amp;gt;http://webdocs.cs.ualberta.ca/~chinook/project/legacy.html&amp;lt;/a&amp;gt;.&#039;&gt;&lt;sup&gt;3&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; Thus progress was definitely beyond beginner level by 1962.&lt;/p&gt; + &lt;/HTML&gt; + + + === Superhuman level === + + + &lt;HTML&gt; + &lt;p&gt;In 1994, computer program Chinook drew six times against world champion Marius Tinsley, before Tinsley withdrew due to pancreatic cancer and Chinook officially won.&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-4-2704&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-4-2704&quot; title=&#039;&amp;amp;#8220;In a rematch, Chinook was declared the Man-Machine World Champion in checkers in 1994 in a match against Marion Tinsley after six drawn games, and Tinsley&amp;amp;#8217;s withdrawal due to pancreatic cancer. While Chinook became the world champion, it never defeated the best checkers player of all time, Tinsley, who was significantly superior to even his closest peer.&amp;lt;sup&amp;gt;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Chinook_(computer_program)#cite_note-1&quot;&amp;gt;[1]&amp;lt;/a&amp;gt;&amp;lt;/sup&amp;gt;&amp;amp;#8220;&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;“Chinook (Computer Program).” In &amp;lt;em&amp;gt;Wikipedia&amp;lt;/em&amp;gt;, August 24, 2020. &amp;lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Chinook_(computer_program)&amp;amp;amp;oldid=974711515&quot;&amp;gt;https://en.wikipedia.org/w/index.php?title=Chinook_(computer_program)&amp;amp;amp;oldid=974711515&amp;lt;/a&amp;gt;.&#039;&gt;&lt;sup&gt;4&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; Thus Chinook appears to have been close to as good as the best player in 1994.&lt;/p&gt; + &lt;/HTML&gt; + + + === End of progress === + + + &lt;HTML&gt; + &lt;p&gt;In 2007 checkers was ‘weakly solved’, which is to say that perfect play guaranteeing a draw for both sides from the start of the game is known, from the starting state&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-5-2704&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-5-2704&quot; title=&#039;&amp;amp;#8220;English draughts was&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game&quot;&amp;gt;weakly solved&amp;lt;/a&amp;gt;&amp;amp;nbsp;in 2007 by the team of Canadian computer scientist&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Jonathan_Schaeffer&quot;&amp;gt;Jonathan Schaeffer&amp;lt;/a&amp;gt;. From the standard starting position, both players can guarantee a draw with&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Perfect_play&quot;&amp;gt;perfect play&amp;lt;/a&amp;gt;.&amp;amp;#8221;&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;“Draughts.” In &amp;lt;em&amp;gt;Wikipedia&amp;lt;/em&amp;gt;, October 25, 2020. &amp;lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Draughts&amp;amp;amp;oldid=985336948&quot;&amp;gt;https://en.wikipedia.org/w/index.php?title=Draughts&amp;amp;amp;oldid=985336948&amp;lt;/a&amp;gt;.&#039;&gt;&lt;sup&gt;5&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; (this does not imply that if someone plays imperfectly, perfect moves following this are known).&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-6-2704&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-6-2704&quot; title=&#039;&amp;amp;#8220;A&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Two-player_game&quot;&amp;gt;two-player game&amp;lt;/a&amp;gt;&amp;amp;nbsp;can be solved on several levels:&amp;lt;sup&amp;gt;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game#cite_note-Allis-1&quot;&amp;gt;[1]&amp;lt;/a&amp;gt;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game#cite_note-2&quot;&amp;gt;[2]&amp;lt;/a&amp;gt;&amp;lt;/sup&amp;gt;&amp;amp;#8230;&amp;lt;br&amp;gt;&amp;lt;strong&amp;gt;Weak&amp;lt;/strong&amp;gt; &amp;lt;br&amp;gt;Provide an algorithm that secures a win for one player, or a draw for either, against any possible moves by the opponent, from the beginning of the game. That is, produce at least one complete ideal game (all moves start to end) with proof that each move is optimal for the player making it. It does not necessarily mean a computer program using the solution will play optimally against an imperfect opponent.&amp;amp;#8221;&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;“Solved Game.” In &amp;lt;em&amp;gt;Wikipedia&amp;lt;/em&amp;gt;, September 11, 2020. &amp;lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Solved_game&amp;amp;amp;oldid=977794097&quot;&amp;gt;https://en.wikipedia.org/w/index.php?title=Solved_game&amp;amp;amp;oldid=977794097&amp;lt;/a&amp;gt;.&#039;&gt;&lt;sup&gt;6&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; This is not the best possible performance by all measures, since further progress could presumably be made on reliably beating worse players.&lt;/p&gt; + &lt;/HTML&gt; + + + ==== Times for AI to cross human-relative ranges  ==== + + + &lt;HTML&gt; + &lt;p&gt;Given the above dates, we have:&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;figure class=&quot;wp-block-table&quot;&gt; + &lt;table&gt; + &lt;tbody&gt; + &lt;tr&gt; + &lt;td&gt;&lt;strong&gt;Range&lt;/strong&gt;&lt;/td&gt; + &lt;td&gt;&lt;strong&gt;Start&lt;/strong&gt;&lt;/td&gt; + &lt;td&gt;&lt;strong&gt;End&lt;/strong&gt;&lt;/td&gt; + &lt;td&gt;&lt;strong&gt;Duration (years)&lt;/strong&gt;&lt;/td&gt; + &lt;/tr&gt; + &lt;tr&gt; + &lt;td&gt;First attempt to beginner level&lt;/td&gt; + &lt;td&gt;1951&lt;/td&gt; + &lt;td&gt;~1956, &amp;lt;1961&lt;/td&gt; + &lt;td&gt;~4, &amp;lt;10&lt;/td&gt; + &lt;/tr&gt; + &lt;tr&gt; + &lt;td&gt;Beginner to superhuman&lt;/td&gt; + &lt;td&gt;~1956, &amp;lt;1961&lt;/td&gt; + &lt;td&gt;1994&lt;/td&gt; + &lt;td&gt;~38, &amp;gt;33&lt;/td&gt; + &lt;/tr&gt; + &lt;tr&gt; + &lt;td&gt;Above superhuman&lt;/td&gt; + &lt;td&gt;1994&lt;/td&gt; + &lt;td&gt;2007*&lt;/td&gt; + &lt;td&gt;13*&lt;/td&gt; + &lt;/tr&gt; + &lt;/tbody&gt; + &lt;/table&gt; + &lt;figcaption&gt; + * treating perfect play as the end of progress, though progress could potentially be made in performing better against imperfect play. + &lt;/figcaption&gt; + &lt;/figure&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;p&gt;&lt;em&gt;&lt;em&gt;Primary author: Katja Grace&lt;/em&gt;&lt;/em&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + ===== Notes ===== + + + &lt;HTML&gt; + &lt;ol class=&quot;easy-footnotes-wrapper&quot;&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-1-2704&quot;&gt;&lt;/span&gt;“The first English draughts &lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_program&quot;&gt;computer program&lt;/a&gt; was written by &lt;a href=&quot;https://en.wikipedia.org/wiki/Christopher_Strachey&quot;&gt;Christopher Strachey&lt;/a&gt;, M.A. at the &lt;a href=&quot;https://en.wikipedia.org/wiki/National_Physical_Laboratory_(United_Kingdom)&quot;&gt;National Physical Laboratory&lt;/a&gt;, London.&lt;sup&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/English_draughts#cite_note-5&quot;&gt;[5]&lt;/a&gt;&lt;/sup&gt; Strachey finished the programme, written in his spare time, in February 1951. It ran for the first time on NPL’s &lt;a href=&quot;https://en.wikipedia.org/wiki/Pilot_ACE&quot;&gt;Pilot ACE&lt;/a&gt; on 30 July 1951. He soon modified the programme to run on the &lt;a href=&quot;https://en.wikipedia.org/wiki/Manchester_Mark_1&quot;&gt;Manchester Mark 1&lt;/a&gt;.” + &lt;p&gt;“English Draughts.” In &lt;em&gt;Wikipedia&lt;/em&gt;, October 8, 2020. &lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=English_draughts&amp;amp;oldid=982525532&quot;&gt;https://en.wikipedia.org/w/index.php?title=English_draughts&amp;amp;oldid=982525532&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-1-2704&quot;&gt;&lt;/a&gt;&lt;/p&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-2-2704&quot;&gt;&lt;/span&gt;“It didn’t take long before Samuel had a program that played a respectable game of checkers, capable of easily defeating novice players. It was first publicly demonstrated on television on February 24, 1956. Thomas Watson, President of IBM, arranged for the program to be exhibited to shareholders. He predicted that it would result in a fifteen-point rise in the IBM stock price. It did.” + &lt;p&gt;Schaeffer, Jonathan. &lt;em&gt;One Jump Ahead: Challenging Human Supremacy in Checkers&lt;/em&gt;. 1st Edition. New York: Springer, 1997.&lt;/p&gt; + &lt;p&gt;quoted in:&lt;/p&gt; + &lt;p&gt;“Legacy – Chinook – World Man-Machine Checkers Champion.” Accessed October 26, 2020. &lt;a href=&quot;http://webdocs.cs.ualberta.ca/~chinook/project/legacy.html&quot;&gt;http://webdocs.cs.ualberta.ca/~chinook/project/legacy.html&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-2-2704&quot;&gt;&lt;/a&gt;&lt;/p&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-3-2704&quot;&gt;&lt;/span&gt;The historical record is fuzzy, but for some reason he chose to have his program debut against Robert Nealey a blind checkers champion from Stamford, Connecticut. IBM’s&lt;em&gt; Research News&lt;/em&gt; claimed that Nealey was “a former Connecticut checkers champion, and one of the nation’s foremost players.” + &lt;p&gt;Although a self-proclaimed master, Nealey’s tournament results never justified such a claim. At the time of the game, he was not a former Connecticut state champion, although he did win the title in 1966, four years after the game with Samuel’s program. In the history of the state champi­onship, no recognized master had won the event. Nealey didn’t play in any of the premier checkers events, such as the U.S. championship, and apparently acquired his reputation by beating local players.”&lt;/p&gt; + &lt;p&gt;Schaeffer, Jonathan. &lt;em&gt;One Jump Ahead: Challenging Human Supremacy in Checkers&lt;/em&gt;. 1st Edition. New York: Springer, 1997.&lt;/p&gt; + &lt;p&gt;quoted in:&lt;/p&gt; + &lt;p&gt;“Legacy – Chinook – World Man-Machine Checkers Champion.” Accessed October 26, 2020. &lt;a href=&quot;http://webdocs.cs.ualberta.ca/~chinook/project/legacy.html&quot;&gt;http://webdocs.cs.ualberta.ca/~chinook/project/legacy.html&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-3-2704&quot;&gt;&lt;/a&gt;&lt;/p&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-4-2704&quot;&gt;&lt;/span&gt;“In a rematch, Chinook was declared the Man-Machine World Champion in checkers in 1994 in a match against Marion Tinsley after six drawn games, and Tinsley’s withdrawal due to pancreatic cancer. While Chinook became the world champion, it never defeated the best checkers player of all time, Tinsley, who was significantly superior to even his closest peer.&lt;sup&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Chinook_(computer_program)#cite_note-1&quot;&gt;[1]&lt;/a&gt;&lt;/sup&gt;“ + &lt;p&gt;“Chinook (Computer Program).” In &lt;em&gt;Wikipedia&lt;/em&gt;, August 24, 2020. &lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Chinook_(computer_program)&amp;amp;oldid=974711515&quot;&gt;https://en.wikipedia.org/w/index.php?title=Chinook_(computer_program)&amp;amp;oldid=974711515&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-4-2704&quot;&gt;&lt;/a&gt;&lt;/p&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-5-2704&quot;&gt;&lt;/span&gt;“English draughts was &lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game&quot;&gt;weakly solved&lt;/a&gt; in 2007 by the team of Canadian computer scientist &lt;a href=&quot;https://en.wikipedia.org/wiki/Jonathan_Schaeffer&quot;&gt;Jonathan Schaeffer&lt;/a&gt;. From the standard starting position, both players can guarantee a draw with &lt;a href=&quot;https://en.wikipedia.org/wiki/Perfect_play&quot;&gt;perfect play&lt;/a&gt;.” + &lt;p&gt;“Draughts.” In &lt;em&gt;Wikipedia&lt;/em&gt;, October 25, 2020. &lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Draughts&amp;amp;oldid=985336948&quot;&gt;https://en.wikipedia.org/w/index.php?title=Draughts&amp;amp;oldid=985336948&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-5-2704&quot;&gt;&lt;/a&gt;&lt;/p&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-6-2704&quot;&gt;&lt;/span&gt;“A &lt;a href=&quot;https://en.wikipedia.org/wiki/Two-player_game&quot;&gt;two-player game&lt;/a&gt; can be solved on several levels:&lt;sup&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game#cite_note-Allis-1&quot;&gt;[1]&lt;/a&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Solved_game#cite_note-2&quot;&gt;[2]&lt;/a&gt;&lt;/sup&gt;…&lt;br/&gt; + &lt;strong&gt;Weak&lt;/strong&gt;&lt;br/&gt; + Provide an algorithm that secures a win for one player, or a draw for either, against any possible moves by the opponent, from the beginning of the game. That is, produce at least one complete ideal game (all moves start to end) with proof that each move is optimal for the player making it. It does not necessarily mean a computer program using the solution will play optimally against an imperfect opponent.” + &lt;p&gt;“Solved Game.” In &lt;em&gt;Wikipedia&lt;/em&gt;, September 11, 2020. &lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Solved_game&amp;amp;oldid=977794097&quot;&gt;https://en.wikipedia.org/w/index.php?title=Solved_game&amp;amp;oldid=977794097&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-6-2704&quot;&gt;&lt;/a&gt;&lt;/p&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;/ol&gt; + &lt;/HTML&gt; + + </pre>

2022-09-21T07:37:41+00:00 2022-09-21T07:37:41+00:00 https://wiki.aiimpacts.org/speed_of_ai_transition/range_of_human_performance/time_for_ai_to_cross_the_human_range_in_starcraft?rev=1663745861&do=diff Anonymous anonymous@undisclosed.example.com <pre> @@ -1 +1,131 @@ + ====== Time for AI to cross the human range in StarCraft ====== + + // Published 20 October, 2020; last updated 08 March, 2021 // + + + + &lt;HTML&gt; + &lt;p&gt;Progress in AI StarCraft performance took:&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;ul&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt;~0 years to reach the level of an untrained human&lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt;~21 years to pass from beginner level to high professional human level&lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt;~2 years to continue from trained human to current performance (2020), with no particular end in sight.&lt;/div&gt;&lt;/li&gt; + &lt;/ul&gt; + &lt;/HTML&gt; + + + + ===== Details ===== + + + ==== Metric ==== + + + &lt;HTML&gt; + &lt;p&gt;We compare human and AI players on their direct ability to beat one another (rather than a measure of the overall performance of each).&lt;/p&gt; + &lt;/HTML&gt; + + + ==== AI milestones ==== + + + === Earliest attempt === + + + &lt;HTML&gt; + &lt;p&gt;Starcraft was released in 1998&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-1-2684&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-1-2684&quot; title=&#039;&amp;amp;#8220;The series debuted with the&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Video_game&quot;&amp;gt;video game&amp;lt;/a&amp;gt;&amp;amp;nbsp;&amp;lt;em&amp;gt;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/StarCraft_(video_game)&quot;&amp;gt;StarCraft&amp;lt;/a&amp;gt;&amp;lt;/em&amp;gt;&amp;amp;nbsp;in 1998.&amp;amp;#8221;&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;“StarCraft.” In &amp;lt;em&amp;gt;Wikipedia&amp;lt;/em&amp;gt;, September 29, 2020. &amp;lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=StarCraft&amp;amp;amp;oldid=980937551&quot;&amp;gt;https://en.wikipedia.org/w/index.php?title=StarCraft&amp;amp;amp;oldid=980937551&amp;lt;/a&amp;gt;.&#039;&gt;&lt;sup&gt;1&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; The game allows the player to play against a computer opponent, however this built-in AI has access to information that a normal player would not have. For instance, it has real-time information about what the other player is doing at all times, which is normally hidden. We do not have detailed knowledge about early StarCraft AIs that do not have this advantage, but our impression is that it was possible to write them from the start (see next section).&lt;/p&gt; + &lt;/HTML&gt; + + + === Beginner level === + + + &lt;HTML&gt; + &lt;p&gt;Our impression is that since StarCraft Brood War came out in 1998, it has been possible to write a bot that can beat a player who recently learned the game, without “cheating” in the way that the game’s built-in computer opponents do. This is uncertain, and based on private discussion with people who write Starcraft AIs that compete in tournaments.&lt;/p&gt; + &lt;/HTML&gt; + + + === Professional level === + + + &lt;HTML&gt; + &lt;p&gt;In 2018, DeepMind’s AlphaStar beat MaNa&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-2-2684&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-2-2684&quot; title=&#039;&amp;amp;#8220;In a series of test matches held on 19 December, AlphaStar decisively beat&amp;amp;nbsp;&amp;lt;a href=&quot;https://www.teamliquid.com/&quot;&amp;gt;Team Liquid’s&amp;lt;/a&amp;gt;&amp;amp;nbsp;Grzegorz &amp;amp;#8220;&amp;lt;a href=&quot;https://liquipedia.net/starcraft2/MaNa&quot;&amp;gt;MaNa&amp;lt;/a&amp;gt;&amp;amp;#8221; Komincz,&amp;amp;nbsp;&amp;lt;a href=&quot;https://liquipedia.net/starcraft2/2018_StarCraft_II_World_Championship_Series_Circuit/Standings&quot;&amp;gt;one of the world’s strongest professional StarCraft players&amp;lt;/a&amp;gt;, 5-0, following a successful benchmark match against his team-mate Dario “&amp;lt;a href=&quot;https://liquipedia.net/starcraft2/TLO&quot;&amp;gt;TLO&amp;lt;/a&amp;gt;” Wünsch. The matches took place under professional match conditions on a competitive ladder&amp;amp;nbsp;&amp;lt;a href=&quot;https://liquipedia.net/starcraft2/Catalyst_LE&quot;&amp;gt;map&amp;lt;/a&amp;gt;&amp;amp;nbsp;and without any game restrictions.&amp;amp;#8221;&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;Deepmind. “AlphaStar: Mastering the Real-Time Strategy Game StarCraft II.” Accessed October 22, 2020. &amp;lt;a href=&quot;https://doi.org//blog/article/alphastar-mastering-real-time-strategy-game-starcraft-ii&quot;&amp;gt;/blog/article/alphastar-mastering-real-time-strategy-game-starcraft-ii&amp;lt;/a&amp;gt;.&#039;&gt;&lt;sup&gt;2&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;, a strong professional player (seemingly 13th place in the 2018 StarCraft II World Championship Series Circuit)&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-3-2684&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-3-2684&quot; title=&#039;Liquipedia StarCraft 2 Wiki. “2018 StarCraft II World Championship Series Circuit: Standings.” Accessed October 20, 2020. &amp;lt;a href=&quot;https://liquipedia.net/starcraft2/2018_StarCraft_II_World_Championship_Series_Circuit/Standings&quot;&amp;gt;https://liquipedia.net/starcraft2/2018_StarCraft_II_World_Championship_Series_Circuit/Standings&amp;lt;/a&amp;gt;.&#039;&gt;&lt;sup&gt;3&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;. This does not imply that AlphaStar was in general a stronger player than MaNa, but suggests AlphaStar was at a broadly high professional level. How AlphaStar’s performance compares to humans will depend on how narrowly the task is defined, so that if the AI is not allowed to give commands faster than a human is able, it will compare less favorably than if it is allowed to give commands very quickly&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-4-2684&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-4-2684&quot; title=&#039;AI Impacts researcher Rick Korzekwa details this controversy in a blog post:&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;“The Unexpected Difficulty of Comparing AlphaStar to Humans – AI Impacts.” Accessed October 22, 2020. &amp;lt;a href=&quot;https://aiimpacts.org/the-unexpected-difficulty-of-comparing-alphastar-to-humans/&quot;&amp;gt;https://aiimpacts.org/the-unexpected-difficulty-of-comparing-alphastar-to-humans/&amp;lt;/a&amp;gt;.&#039;&gt;&lt;sup&gt;4&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;.&lt;/p&gt; + &lt;/HTML&gt; + + + ==== Times for AI to cross human-relative ranges  ==== + + + &lt;HTML&gt; + &lt;p&gt;Given the above dates, we have:&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;figure class=&quot;wp-block-table&quot;&gt; + &lt;table&gt; + &lt;tbody&gt; + &lt;tr&gt; + &lt;td&gt;&lt;strong&gt;Range&lt;/strong&gt;&lt;/td&gt; + &lt;td&gt;&lt;strong&gt;Start&lt;/strong&gt;&lt;/td&gt; + &lt;td&gt;&lt;strong&gt;End&lt;/strong&gt;&lt;/td&gt; + &lt;td&gt;&lt;strong&gt;Duration (years)&lt;/strong&gt;&lt;/td&gt; + &lt;/tr&gt; + &lt;tr&gt; + &lt;td&gt;First attempt to beginner level&lt;/td&gt; + &lt;td&gt;1998&lt;/td&gt; + &lt;td&gt;1998&lt;/td&gt; + &lt;td&gt;~0&lt;/td&gt; + &lt;/tr&gt; + &lt;tr&gt; + &lt;td&gt;Beginner to superhuman&lt;/td&gt; + &lt;td&gt;1998&lt;/td&gt; + &lt;td&gt;2018&lt;/td&gt; + &lt;td&gt;~21&lt;/td&gt; + &lt;/tr&gt; + &lt;tr&gt; + &lt;td&gt;Above superhuman&lt;/td&gt; + &lt;td&gt;2018&lt;/td&gt; + &lt;td&gt;?&lt;/td&gt; + &lt;td&gt;&amp;gt;2&lt;/td&gt; + &lt;/tr&gt; + &lt;/tbody&gt; + &lt;/table&gt; + &lt;/figure&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;p&gt;&lt;em&gt;Primary author: Rick Korzekwa&lt;/em&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + ===== Notes ===== + + + &lt;HTML&gt; + &lt;ol class=&quot;easy-footnotes-wrapper&quot;&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-1-2684&quot;&gt;&lt;/span&gt;“The series debuted with the &lt;a href=&quot;https://en.wikipedia.org/wiki/Video_game&quot;&gt;video game&lt;/a&gt; &lt;em&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/StarCraft_(video_game)&quot;&gt;StarCraft&lt;/a&gt;&lt;/em&gt; in 1998.” + &lt;p&gt;“StarCraft.” In &lt;em&gt;Wikipedia&lt;/em&gt;, September 29, 2020. &lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=StarCraft&amp;amp;oldid=980937551&quot;&gt;https://en.wikipedia.org/w/index.php?title=StarCraft&amp;amp;oldid=980937551&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-1-2684&quot;&gt;&lt;/a&gt;&lt;/p&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-2-2684&quot;&gt;&lt;/span&gt;“In a series of test matches held on 19 December, AlphaStar decisively beat &lt;a href=&quot;https://www.teamliquid.com/&quot;&gt;Team Liquid’s&lt;/a&gt; Grzegorz “&lt;a href=&quot;https://liquipedia.net/starcraft2/MaNa&quot;&gt;MaNa&lt;/a&gt;” Komincz, &lt;a href=&quot;https://liquipedia.net/starcraft2/2018_StarCraft_II_World_Championship_Series_Circuit/Standings&quot;&gt;one of the world’s strongest professional StarCraft players&lt;/a&gt;, 5-0, following a successful benchmark match against his team-mate Dario “&lt;a href=&quot;https://liquipedia.net/starcraft2/TLO&quot;&gt;TLO&lt;/a&gt;” Wünsch. The matches took place under professional match conditions on a competitive ladder &lt;a href=&quot;https://liquipedia.net/starcraft2/Catalyst_LE&quot;&gt;map&lt;/a&gt; and without any game restrictions.” + &lt;p&gt;Deepmind. “AlphaStar: Mastering the Real-Time Strategy Game StarCraft II.” Accessed October 22, 2020. &lt;a href=&quot;https://doi.org//blog/article/alphastar-mastering-real-time-strategy-game-starcraft-ii&quot;&gt;/blog/article/alphastar-mastering-real-time-strategy-game-starcraft-ii&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-2-2684&quot;&gt;&lt;/a&gt;&lt;/p&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-3-2684&quot;&gt;&lt;/span&gt;Liquipedia StarCraft 2 Wiki. “2018 StarCraft II World Championship Series Circuit: Standings.” Accessed October 20, 2020. &lt;a href=&quot;https://liquipedia.net/starcraft2/2018_StarCraft_II_World_Championship_Series_Circuit/Standings&quot;&gt;https://liquipedia.net/starcraft2/2018_StarCraft_II_World_Championship_Series_Circuit/Standings&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-3-2684&quot;&gt;&lt;/a&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-4-2684&quot;&gt;&lt;/span&gt;AI Impacts researcher Rick Korzekwa details this controversy in a blog post: + &lt;p&gt;“The Unexpected Difficulty of Comparing AlphaStar to Humans – AI Impacts.” Accessed October 22, 2020. &lt;a href=&quot;https://aiimpacts.org/the-unexpected-difficulty-of-comparing-alphastar-to-humans/&quot;&gt;https://aiimpacts.org/the-unexpected-difficulty-of-comparing-alphastar-to-humans/&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-4-2684&quot;&gt;&lt;/a&gt;&lt;/p&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;/ol&gt; + &lt;/HTML&gt; + + </pre>

<pre> @@ -1 +1,131 @@ + ====== Time for AI to cross the human range in StarCraft ====== + + // Published 20 October, 2020; last updated 08 March, 2021 // + + + + &lt;HTML&gt; + &lt;p&gt;Progress in AI StarCraft performance took:&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;ul&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt;~0 years to reach the level of an untrained human&lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt;~21 years to pass from beginner level to high professional human level&lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt;~2 years to continue from trained human to current performance (2020), with no particular end in sight.&lt;/div&gt;&lt;/li&gt; + &lt;/ul&gt; + &lt;/HTML&gt; + + + + ===== Details ===== + + + ==== Metric ==== + + + &lt;HTML&gt; + &lt;p&gt;We compare human and AI players on their direct ability to beat one another (rather than a measure of the overall performance of each).&lt;/p&gt; + &lt;/HTML&gt; + + + ==== AI milestones ==== + + + === Earliest attempt === + + + &lt;HTML&gt; + &lt;p&gt;Starcraft was released in 1998&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-1-2684&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-1-2684&quot; title=&#039;&amp;amp;#8220;The series debuted with the&amp;amp;nbsp;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/Video_game&quot;&amp;gt;video game&amp;lt;/a&amp;gt;&amp;amp;nbsp;&amp;lt;em&amp;gt;&amp;lt;a href=&quot;https://en.wikipedia.org/wiki/StarCraft_(video_game)&quot;&amp;gt;StarCraft&amp;lt;/a&amp;gt;&amp;lt;/em&amp;gt;&amp;amp;nbsp;in 1998.&amp;amp;#8221;&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;“StarCraft.” In &amp;lt;em&amp;gt;Wikipedia&amp;lt;/em&amp;gt;, September 29, 2020. &amp;lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=StarCraft&amp;amp;amp;oldid=980937551&quot;&amp;gt;https://en.wikipedia.org/w/index.php?title=StarCraft&amp;amp;amp;oldid=980937551&amp;lt;/a&amp;gt;.&#039;&gt;&lt;sup&gt;1&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; The game allows the player to play against a computer opponent, however this built-in AI has access to information that a normal player would not have. For instance, it has real-time information about what the other player is doing at all times, which is normally hidden. We do not have detailed knowledge about early StarCraft AIs that do not have this advantage, but our impression is that it was possible to write them from the start (see next section).&lt;/p&gt; + &lt;/HTML&gt; + + + === Beginner level === + + + &lt;HTML&gt; + &lt;p&gt;Our impression is that since StarCraft Brood War came out in 1998, it has been possible to write a bot that can beat a player who recently learned the game, without “cheating” in the way that the game’s built-in computer opponents do. This is uncertain, and based on private discussion with people who write Starcraft AIs that compete in tournaments.&lt;/p&gt; + &lt;/HTML&gt; + + + === Professional level === + + + &lt;HTML&gt; + &lt;p&gt;In 2018, DeepMind’s AlphaStar beat MaNa&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-2-2684&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-2-2684&quot; title=&#039;&amp;amp;#8220;In a series of test matches held on 19 December, AlphaStar decisively beat&amp;amp;nbsp;&amp;lt;a href=&quot;https://www.teamliquid.com/&quot;&amp;gt;Team Liquid’s&amp;lt;/a&amp;gt;&amp;amp;nbsp;Grzegorz &amp;amp;#8220;&amp;lt;a href=&quot;https://liquipedia.net/starcraft2/MaNa&quot;&amp;gt;MaNa&amp;lt;/a&amp;gt;&amp;amp;#8221; Komincz,&amp;amp;nbsp;&amp;lt;a href=&quot;https://liquipedia.net/starcraft2/2018_StarCraft_II_World_Championship_Series_Circuit/Standings&quot;&amp;gt;one of the world’s strongest professional StarCraft players&amp;lt;/a&amp;gt;, 5-0, following a successful benchmark match against his team-mate Dario “&amp;lt;a href=&quot;https://liquipedia.net/starcraft2/TLO&quot;&amp;gt;TLO&amp;lt;/a&amp;gt;” Wünsch. The matches took place under professional match conditions on a competitive ladder&amp;amp;nbsp;&amp;lt;a href=&quot;https://liquipedia.net/starcraft2/Catalyst_LE&quot;&amp;gt;map&amp;lt;/a&amp;gt;&amp;amp;nbsp;and without any game restrictions.&amp;amp;#8221;&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;Deepmind. “AlphaStar: Mastering the Real-Time Strategy Game StarCraft II.” Accessed October 22, 2020. &amp;lt;a href=&quot;https://doi.org//blog/article/alphastar-mastering-real-time-strategy-game-starcraft-ii&quot;&amp;gt;/blog/article/alphastar-mastering-real-time-strategy-game-starcraft-ii&amp;lt;/a&amp;gt;.&#039;&gt;&lt;sup&gt;2&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;, a strong professional player (seemingly 13th place in the 2018 StarCraft II World Championship Series Circuit)&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-3-2684&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-3-2684&quot; title=&#039;Liquipedia StarCraft 2 Wiki. “2018 StarCraft II World Championship Series Circuit: Standings.” Accessed October 20, 2020. &amp;lt;a href=&quot;https://liquipedia.net/starcraft2/2018_StarCraft_II_World_Championship_Series_Circuit/Standings&quot;&amp;gt;https://liquipedia.net/starcraft2/2018_StarCraft_II_World_Championship_Series_Circuit/Standings&amp;lt;/a&amp;gt;.&#039;&gt;&lt;sup&gt;3&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;. This does not imply that AlphaStar was in general a stronger player than MaNa, but suggests AlphaStar was at a broadly high professional level. How AlphaStar’s performance compares to humans will depend on how narrowly the task is defined, so that if the AI is not allowed to give commands faster than a human is able, it will compare less favorably than if it is allowed to give commands very quickly&lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-4-2684&quot;&gt;&lt;/span&gt;&lt;span class=&quot;easy-footnote&quot;&gt;&lt;a href=&quot;#easy-footnote-bottom-4-2684&quot; title=&#039;AI Impacts researcher Rick Korzekwa details this controversy in a blog post:&amp;lt;/p&amp;gt; &amp;lt;p&amp;gt;“The Unexpected Difficulty of Comparing AlphaStar to Humans – AI Impacts.” Accessed October 22, 2020. &amp;lt;a href=&quot;https://aiimpacts.org/the-unexpected-difficulty-of-comparing-alphastar-to-humans/&quot;&amp;gt;https://aiimpacts.org/the-unexpected-difficulty-of-comparing-alphastar-to-humans/&amp;lt;/a&amp;gt;.&#039;&gt;&lt;sup&gt;4&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;.&lt;/p&gt; + &lt;/HTML&gt; + + + ==== Times for AI to cross human-relative ranges  ==== + + + &lt;HTML&gt; + &lt;p&gt;Given the above dates, we have:&lt;/p&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;figure class=&quot;wp-block-table&quot;&gt; + &lt;table&gt; + &lt;tbody&gt; + &lt;tr&gt; + &lt;td&gt;&lt;strong&gt;Range&lt;/strong&gt;&lt;/td&gt; + &lt;td&gt;&lt;strong&gt;Start&lt;/strong&gt;&lt;/td&gt; + &lt;td&gt;&lt;strong&gt;End&lt;/strong&gt;&lt;/td&gt; + &lt;td&gt;&lt;strong&gt;Duration (years)&lt;/strong&gt;&lt;/td&gt; + &lt;/tr&gt; + &lt;tr&gt; + &lt;td&gt;First attempt to beginner level&lt;/td&gt; + &lt;td&gt;1998&lt;/td&gt; + &lt;td&gt;1998&lt;/td&gt; + &lt;td&gt;~0&lt;/td&gt; + &lt;/tr&gt; + &lt;tr&gt; + &lt;td&gt;Beginner to superhuman&lt;/td&gt; + &lt;td&gt;1998&lt;/td&gt; + &lt;td&gt;2018&lt;/td&gt; + &lt;td&gt;~21&lt;/td&gt; + &lt;/tr&gt; + &lt;tr&gt; + &lt;td&gt;Above superhuman&lt;/td&gt; + &lt;td&gt;2018&lt;/td&gt; + &lt;td&gt;?&lt;/td&gt; + &lt;td&gt;&amp;gt;2&lt;/td&gt; + &lt;/tr&gt; + &lt;/tbody&gt; + &lt;/table&gt; + &lt;/figure&gt; + &lt;/HTML&gt; + + + &lt;HTML&gt; + &lt;p&gt;&lt;em&gt;Primary author: Rick Korzekwa&lt;/em&gt;&lt;/p&gt; + &lt;/HTML&gt; + + + ===== Notes ===== + + + &lt;HTML&gt; + &lt;ol class=&quot;easy-footnotes-wrapper&quot;&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-1-2684&quot;&gt;&lt;/span&gt;“The series debuted with the &lt;a href=&quot;https://en.wikipedia.org/wiki/Video_game&quot;&gt;video game&lt;/a&gt; &lt;em&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/StarCraft_(video_game)&quot;&gt;StarCraft&lt;/a&gt;&lt;/em&gt; in 1998.” + &lt;p&gt;“StarCraft.” In &lt;em&gt;Wikipedia&lt;/em&gt;, September 29, 2020. &lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=StarCraft&amp;amp;oldid=980937551&quot;&gt;https://en.wikipedia.org/w/index.php?title=StarCraft&amp;amp;oldid=980937551&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-1-2684&quot;&gt;&lt;/a&gt;&lt;/p&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-2-2684&quot;&gt;&lt;/span&gt;“In a series of test matches held on 19 December, AlphaStar decisively beat &lt;a href=&quot;https://www.teamliquid.com/&quot;&gt;Team Liquid’s&lt;/a&gt; Grzegorz “&lt;a href=&quot;https://liquipedia.net/starcraft2/MaNa&quot;&gt;MaNa&lt;/a&gt;” Komincz, &lt;a href=&quot;https://liquipedia.net/starcraft2/2018_StarCraft_II_World_Championship_Series_Circuit/Standings&quot;&gt;one of the world’s strongest professional StarCraft players&lt;/a&gt;, 5-0, following a successful benchmark match against his team-mate Dario “&lt;a href=&quot;https://liquipedia.net/starcraft2/TLO&quot;&gt;TLO&lt;/a&gt;” Wünsch. The matches took place under professional match conditions on a competitive ladder &lt;a href=&quot;https://liquipedia.net/starcraft2/Catalyst_LE&quot;&gt;map&lt;/a&gt; and without any game restrictions.” + &lt;p&gt;Deepmind. “AlphaStar: Mastering the Real-Time Strategy Game StarCraft II.” Accessed October 22, 2020. &lt;a href=&quot;https://doi.org//blog/article/alphastar-mastering-real-time-strategy-game-starcraft-ii&quot;&gt;/blog/article/alphastar-mastering-real-time-strategy-game-starcraft-ii&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-2-2684&quot;&gt;&lt;/a&gt;&lt;/p&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-3-2684&quot;&gt;&lt;/span&gt;Liquipedia StarCraft 2 Wiki. “2018 StarCraft II World Championship Series Circuit: Standings.” Accessed October 20, 2020. &lt;a href=&quot;https://liquipedia.net/starcraft2/2018_StarCraft_II_World_Championship_Series_Circuit/Standings&quot;&gt;https://liquipedia.net/starcraft2/2018_StarCraft_II_World_Championship_Series_Circuit/Standings&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-3-2684&quot;&gt;&lt;/a&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;li&gt;&lt;div class=&quot;li&quot;&gt; + &lt;span class=&quot;easy-footnote-margin-adjust&quot; id=&quot;easy-footnote-bottom-4-2684&quot;&gt;&lt;/span&gt;AI Impacts researcher Rick Korzekwa details this controversy in a blog post: + &lt;p&gt;“The Unexpected Difficulty of Comparing AlphaStar to Humans – AI Impacts.” Accessed October 22, 2020. &lt;a href=&quot;https://aiimpacts.org/the-unexpected-difficulty-of-comparing-alphastar-to-humans/&quot;&gt;https://aiimpacts.org/the-unexpected-difficulty-of-comparing-alphastar-to-humans/&lt;/a&gt;.&lt;a class=&quot;easy-footnote-to-top&quot; href=&quot;#easy-footnote-4-2684&quot;&gt;&lt;/a&gt;&lt;/p&gt; + &lt;/div&gt;&lt;/li&gt; + &lt;/ol&gt; + &lt;/HTML&gt; + + </pre>