Published 07 February, 2020; last updated 27 May, 2020
The Elo rating of the best chess program measured by the Swedish Chess Computer Association did not contain any greater than 10-year discontinuities between 1984 and 2018. A four year discontinuity in 2008 was notable in the context of otherwise regular progress.
This case study is part of AI Impacts’ discontinuous progress investigation.
The history of chess-playing computers is long and rich, partly because chess-playing ability has long been thought (by some) to be a sign of general intelligence.1 The first two ‘chess-playing machines’ were in fact fakes, with small human chess-players crouching inside.2 It was not until 1951 that a program was published (by Alan Turing) that could actually play the full game.3 There has been fairly regular progress since then.4
In 1997 IBM’s chess machine Deep Blue beat Gary Kasparov, world chess champion at the time, under standard tournament time controls.5 This was seen as particularly significant in light of the continued popular association between chess AI and general AI.6 The event marked the point at which chess AI became superhuman, and received substantial press coverage.7
The Swedish Chess Computer Association (SSDF) measures computer chess software performance by playing chess programs against one another on standard hardware.8
According to Wikipedia10:
The Swedish Chess Computer Association (Swedish: Svenska schackdatorföreningen, SSDF) is an organization that tests computer chess software by playing chess programs against one another and producing a rating list…The SSDF list is one of the only statistically significant measures of chess engine strength, especially compared to tournaments, because it incorporates the results of thousands of games played on standard hardware at tournament time controls. The list reports not only absolute rating, but also error bars, winning percentages, and recorded moves of played games.
Looking at the data, we assume a linear trend in Elo.12 There are no discontinuities of 10 or more years.
= Minor discontinuity =
There is a four year discontinuity in 2008. While this is below the scale of interest for our discontinuous progress investigation, it strikes us as notable in the context of otherwise very regular progress.13 We’ve tabulated a number of other potentially relevant metrics for this discontinuity in the ‘Notable discontinuities less than 10 years’ tab here.14
This jump appears to have been partially caused by the introduction of new hardware in the contest, as well as software progress.15
Another example: The tenth Turing Lecture, available here, mentions chess 20 times and uses it as a central example of how the field of artificial intelligence has progressed over the years. Newell, Allen, and Herbert A. Simon. “Computer Science as Empirical Inquiry: Symbols and Search.” ACM Turing Award Lectures: 1975. doi:10.1145/1283920.1283930.