|
|
|
|
— |
speed_of_ai_transition:range_of_human_performance:time_for_ai_to_cross_the_human_range_in_starcraft [2022/09/21 07:37] (current)
|
| | ====== Time for AI to cross the human range in StarCraft ====== |
| | |
| | // Published 20 October, 2020; last updated 08 March, 2021 // |
| | |
| | |
| | |
| | <HTML> |
| | <p>Progress in AI StarCraft performance took:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <ul> |
| | <li><div class="li">~0 years to reach the level of an untrained human</div></li> |
| | <li><div class="li">~21 years to pass from beginner level to high professional human level</div></li> |
| | <li><div class="li">~2 years to continue from trained human to current performance (2020), with no particular end in sight.</div></li> |
| | </ul> |
| | </HTML> |
| | |
| | |
| | |
| | ===== Details ===== |
| | |
| | |
| | ==== Metric ==== |
| | |
| | |
| | <HTML> |
| | <p>We compare human and AI players on their direct ability to beat one another (rather than a measure of the overall performance of each).</p> |
| | </HTML> |
| | |
| | |
| | ==== AI milestones ==== |
| | |
| | |
| | === Earliest attempt === |
| | |
| | |
| | <HTML> |
| | <p>Starcraft was released in 1998<span class="easy-footnote-margin-adjust" id="easy-footnote-1-2684"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-1-2684" title='&#8220;The series debuted with the&nbsp;<a href="https://en.wikipedia.org/wiki/Video_game">video game</a>&nbsp;<em><a href="https://en.wikipedia.org/wiki/StarCraft_(video_game)">StarCraft</a></em>&nbsp;in 1998.&#8221;</p> <p>“StarCraft.” In <em>Wikipedia</em>, September 29, 2020. <a href="https://en.wikipedia.org/w/index.php?title=StarCraft&amp;oldid=980937551">https://en.wikipedia.org/w/index.php?title=StarCraft&amp;oldid=980937551</a>.'><sup>1</sup></a></span> The game allows the player to play against a computer opponent, however this built-in AI has access to information that a normal player would not have. For instance, it has real-time information about what the other player is doing at all times, which is normally hidden. We do not have detailed knowledge about early StarCraft AIs that do not have this advantage, but our impression is that it was possible to write them from the start (see next section).</p> |
| | </HTML> |
| | |
| | |
| | === Beginner level === |
| | |
| | |
| | <HTML> |
| | <p>Our impression is that since StarCraft Brood War came out in 1998, it has been possible to write a bot that can beat a player who recently learned the game, without “cheating” in the way that the game’s built-in computer opponents do. This is uncertain, and based on private discussion with people who write Starcraft AIs that compete in tournaments.</p> |
| | </HTML> |
| | |
| | |
| | === Professional level === |
| | |
| | |
| | <HTML> |
| | <p>In 2018, DeepMind’s AlphaStar beat MaNa<span class="easy-footnote-margin-adjust" id="easy-footnote-2-2684"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-2-2684" title='&#8220;In a series of test matches held on 19 December, AlphaStar decisively beat&nbsp;<a href="https://www.teamliquid.com/">Team Liquid’s</a>&nbsp;Grzegorz &#8220;<a href="https://liquipedia.net/starcraft2/MaNa">MaNa</a>&#8221; Komincz,&nbsp;<a href="https://liquipedia.net/starcraft2/2018_StarCraft_II_World_Championship_Series_Circuit/Standings">one of the world’s strongest professional StarCraft players</a>, 5-0, following a successful benchmark match against his team-mate Dario “<a href="https://liquipedia.net/starcraft2/TLO">TLO</a>” Wünsch. The matches took place under professional match conditions on a competitive ladder&nbsp;<a href="https://liquipedia.net/starcraft2/Catalyst_LE">map</a>&nbsp;and without any game restrictions.&#8221;</p> <p>Deepmind. “AlphaStar: Mastering the Real-Time Strategy Game StarCraft II.” Accessed October 22, 2020. <a href="https://doi.org//blog/article/alphastar-mastering-real-time-strategy-game-starcraft-ii">/blog/article/alphastar-mastering-real-time-strategy-game-starcraft-ii</a>.'><sup>2</sup></a></span>, a strong professional player (seemingly 13th place in the 2018 StarCraft II World Championship Series Circuit)<span class="easy-footnote-margin-adjust" id="easy-footnote-3-2684"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-3-2684" title='Liquipedia StarCraft 2 Wiki. “2018 StarCraft II World Championship Series Circuit: Standings.” Accessed October 20, 2020. <a href="https://liquipedia.net/starcraft2/2018_StarCraft_II_World_Championship_Series_Circuit/Standings">https://liquipedia.net/starcraft2/2018_StarCraft_II_World_Championship_Series_Circuit/Standings</a>.'><sup>3</sup></a></span>. This does not imply that AlphaStar was in general a stronger player than MaNa, but suggests AlphaStar was at a broadly high professional level. How AlphaStar’s performance compares to humans will depend on how narrowly the task is defined, so that if the AI is not allowed to give commands faster than a human is able, it will compare less favorably than if it is allowed to give commands very quickly<span class="easy-footnote-margin-adjust" id="easy-footnote-4-2684"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-4-2684" title='AI Impacts researcher Rick Korzekwa details this controversy in a blog post:</p> <p>“The Unexpected Difficulty of Comparing AlphaStar to Humans – AI Impacts.” Accessed October 22, 2020. <a href="https://aiimpacts.org/the-unexpected-difficulty-of-comparing-alphastar-to-humans/">https://aiimpacts.org/the-unexpected-difficulty-of-comparing-alphastar-to-humans/</a>.'><sup>4</sup></a></span>.</p> |
| | </HTML> |
| | |
| | |
| | ==== Times for AI to cross human-relative ranges ==== |
| | |
| | |
| | <HTML> |
| | <p>Given the above dates, we have:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <figure class="wp-block-table"> |
| | <table> |
| | <tbody> |
| | <tr> |
| | <td><strong>Range</strong></td> |
| | <td><strong>Start</strong></td> |
| | <td><strong>End</strong></td> |
| | <td><strong>Duration (years)</strong></td> |
| | </tr> |
| | <tr> |
| | <td>First attempt to beginner level</td> |
| | <td>1998</td> |
| | <td>1998</td> |
| | <td>~0</td> |
| | </tr> |
| | <tr> |
| | <td>Beginner to superhuman</td> |
| | <td>1998</td> |
| | <td>2018</td> |
| | <td>~21</td> |
| | </tr> |
| | <tr> |
| | <td>Above superhuman</td> |
| | <td>2018</td> |
| | <td>?</td> |
| | <td>>2</td> |
| | </tr> |
| | </tbody> |
| | </table> |
| | </figure> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p><em>Primary author: Rick Korzekwa</em></p> |
| | </HTML> |
| | |
| | |
| | ===== Notes ===== |
| | |
| | |
| | <HTML> |
| | <ol class="easy-footnotes-wrapper"> |
| | <li><div class="li"> |
| | <span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-1-2684"></span>“The series debuted with the <a href="https://en.wikipedia.org/wiki/Video_game">video game</a> <em><a href="https://en.wikipedia.org/wiki/StarCraft_(video_game)">StarCraft</a></em> in 1998.” |
| | <p>“StarCraft.” In <em>Wikipedia</em>, September 29, 2020. <a href="https://en.wikipedia.org/w/index.php?title=StarCraft&oldid=980937551">https://en.wikipedia.org/w/index.php?title=StarCraft&oldid=980937551</a>.<a class="easy-footnote-to-top" href="#easy-footnote-1-2684"></a></p> |
| | </div></li> |
| | <li><div class="li"> |
| | <span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-2-2684"></span>“In a series of test matches held on 19 December, AlphaStar decisively beat <a href="https://www.teamliquid.com/">Team Liquid’s</a> Grzegorz “<a href="https://liquipedia.net/starcraft2/MaNa">MaNa</a>” Komincz, <a href="https://liquipedia.net/starcraft2/2018_StarCraft_II_World_Championship_Series_Circuit/Standings">one of the world’s strongest professional StarCraft players</a>, 5-0, following a successful benchmark match against his team-mate Dario “<a href="https://liquipedia.net/starcraft2/TLO">TLO</a>” Wünsch. The matches took place under professional match conditions on a competitive ladder <a href="https://liquipedia.net/starcraft2/Catalyst_LE">map</a> and without any game restrictions.” |
| | <p>Deepmind. “AlphaStar: Mastering the Real-Time Strategy Game StarCraft II.” Accessed October 22, 2020. <a href="https://doi.org//blog/article/alphastar-mastering-real-time-strategy-game-starcraft-ii">/blog/article/alphastar-mastering-real-time-strategy-game-starcraft-ii</a>.<a class="easy-footnote-to-top" href="#easy-footnote-2-2684"></a></p> |
| | </div></li> |
| | <li><div class="li"> |
| | <span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-3-2684"></span>Liquipedia StarCraft 2 Wiki. “2018 StarCraft II World Championship Series Circuit: Standings.” Accessed October 20, 2020. <a href="https://liquipedia.net/starcraft2/2018_StarCraft_II_World_Championship_Series_Circuit/Standings">https://liquipedia.net/starcraft2/2018_StarCraft_II_World_Championship_Series_Circuit/Standings</a>.<a class="easy-footnote-to-top" href="#easy-footnote-3-2684"></a> |
| | </div></li> |
| | <li><div class="li"> |
| | <span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-4-2684"></span>AI Impacts researcher Rick Korzekwa details this controversy in a blog post: |
| | <p>“The Unexpected Difficulty of Comparing AlphaStar to Humans – AI Impacts.” Accessed October 22, 2020. <a href="https://aiimpacts.org/the-unexpected-difficulty-of-comparing-alphastar-to-humans/">https://aiimpacts.org/the-unexpected-difficulty-of-comparing-alphastar-to-humans/</a>.<a class="easy-footnote-to-top" href="#easy-footnote-4-2684"></a></p> |
| | </div></li> |
| | </ol> |
| | </HTML> |
| | |
| | |
| |
