|
|
|
|
— |
ai_timelines:trends_in_the_cost_of_computing [2022/09/21 07:37] (current)
|
| | ====== Trends in the cost of computing ====== |
| | |
| | // Published 10 March, 2015; last updated 11 June, 2022 // |
| | |
| | |
| | |
| | <HTML> |
| | <p>Computing power available per dollar has probably increased by a factor of ten roughly every four years over the last quarter of a century (measured in FLOPS or MIPS).</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>Over the past 6-8 years, the rate has been slower: around an order of magnitude every 10-16 years, measured in single precision theoretical peak FLOPS or Passmark’s benchmark scores.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>Since the 1940s, MIPS/$ have grown by a factor of ten roughly every five years, and FLOPS/$ roughly every 7.7 years.</p> |
| | </HTML> |
| | |
| | |
| | |
| | ===== Evidence ===== |
| | |
| | |
| | ==== Nordhaus ==== |
| | |
| | |
| | <HTML> |
| | <p><a href="https://web.archive.org/web/20160222082744/http://www.econ.yale.edu/~nordhaus/homepage/prog_083001a.pdf">Nordhaus (2001)</a> analyzes the cost of computing over the past century and a half, and produces Figure 1 (though the scale on the vertical axis appears to be off by many orders of magnitude). Much of his data comes from Moravec’s <em><a href="http://books.google.com/books/about/Mind_Children.html?id=56mb7XuSx3QC">Mind Children</a> </em>(an updated version of the data is <a href="https://web.archive.org/web/20161112110101/http://www.transhumanist.com:80/volume1/moravec.htm">here</a>). He converts all data points to ‘million standard operations per second’ (MSOPS), where a standard operation is a weighted mixture of multiplications and additions. He says it is approximately equivalent to 1 MIPS under the Dhrystone metric.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>He calculates that performance improved at an average rate of 55% per year since 1940. That is, an order of magnitude roughly every five years. However he finds that the average growth rate in different decades differed markedly, with growth since 1980 (until writing in 2001) at around 80% per year, and growth in the 60s and 70s at less than 30% (see figure 2). This would correspond to improving by an order of magnitude every four years in the 80s and 90s.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <figure aria-describedby="caption-attachment-449" class="wp-caption alignnone" id="attachment_449" style="width: 600px"> |
| | <a href="http://aiimpacts.org/wp-content/uploads/2015/03/nordhauscomp-copy.png"><img alt='"The progress of computing measured in cost per million standardized operations per second (MSOPS) deflated by the consumer price index." (From Figure 1, Nordhaus, 2001)' class="wp-image-449" height="469" sizes="(max-width: 600px) 100vw, 600px" src="https://aiimpacts.org/wp-content/uploads/2015/03/nordhauscomp-copy.png" srcset="https://aiimpacts.org/wp-content/uploads/2015/03/nordhauscomp-copy.png 703w, https://aiimpacts.org/wp-content/uploads/2015/03/nordhauscomp-copy-300x234.png 300w" width="600"/></a> |
| | <figcaption class="wp-caption-text" id="caption-attachment-449"> |
| | <strong>Figure 1:</strong> “The progress of computing measured in cost per million standardized operations per second (MSOPS) deflated by the consumer price index.” <strong>Note that the vertical axis appears to be mislabeled—the scale is around seven orders of magnitude different from other sources, such as <a href="https://web.archive.org/web/20161112110101/http://www.transhumanist.com:80/volume1/moravec.htm">Moravec</a>.</strong> (From Figure 1, <a href="https://web.archive.org/web/20160222082744/http://www.econ.yale.edu/~nordhaus/homepage/prog_083001a.pdf">Nordhaus, 2001</a>, p38) |
| | </figcaption> |
| | </figure> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <figure aria-describedby="caption-attachment-452" class="wp-caption alignnone" id="attachment_452" style="width: 600px"> |
| | <a href="http://aiimpacts.org/wp-content/uploads/2015/03/nordhausdecades-copy.png"><img alt="Figure xxx: " class="wp-image-452" height="503" loading="lazy" sizes="(max-width: 600px) 100vw, 600px" src="https://aiimpacts.org/wp-content/uploads/2015/03/nordhausdecades-copy-1024x859.png" srcset="https://aiimpacts.org/wp-content/uploads/2015/03/nordhausdecades-copy-1024x859.png 1024w, https://aiimpacts.org/wp-content/uploads/2015/03/nordhausdecades-copy-300x252.png 300w, https://aiimpacts.org/wp-content/uploads/2015/03/nordhausdecades-copy.png 1442w" width="600"/></a> |
| | <figcaption class="wp-caption-text" id="caption-attachment-452"> |
| | <strong>Figure 2:</strong> From Nordhaus p42,”Rate of Growth of Computer Power by Epoch…Real computer power is the inverse of the decline of real computation costs…” |
| | </figcaption> |
| | </figure> |
| | </HTML> |
| | |
| | |
| | ==== Sandberg and Bostrom ==== |
| | |
| | |
| | <HTML> |
| | <p><a href="http://www.fhi.ox.ac.uk/brain-emulation-roadmap-report.pdf" rel="nofollow">Sandberg and Bostrom (2008)</a> investigate hardware performance trends in their Whole Brain Emulation Roadmap (Appendix B). They plot price performance in MIPS/$ and FLOPS/$, as shown in Figures 3 and 4. They find MIPS/$ grows by a factor of ten every 5.6 years (with a <a href="http://en.wikipedia.org/wiki/Bootstrapping_%28statistics%29">bootstrap</a> 95% confidence interval of 5.3-5.9), and FLOPs/$ grows by a factor of ten every 7.7 years (with a bootstrap confidence interval of 6.5‐9.2 years).</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>They find that growth in MIPS/$ slowed in the 70s and 80s, then accelerated again (most recently gaining an order of magnitude every 3.5 years), which is close to what Nordhaus found.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>Sandberg and Bostrom’s data is from John McCallum’s <a href="http://www.jcmit.com/cpu-performance.htm">CPU price performance dataset,</a> which does not appear to draw directly from Moravec’s data.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <figure aria-describedby="caption-attachment-453" class="wp-caption alignnone" id="attachment_453" style="width: 600px"> |
| | <a href="http://aiimpacts.org/wp-content/uploads/2015/03/wberm-mips-copy.jpg"><img alt="Figure xxx:" class="wp-image-453" height="490" loading="lazy" sizes="(max-width: 600px) 100vw, 600px" src="https://aiimpacts.org/wp-content/uploads/2015/03/wberm-mips-copy.jpg" srcset="https://aiimpacts.org/wp-content/uploads/2015/03/wberm-mips-copy.jpg 1010w, https://aiimpacts.org/wp-content/uploads/2015/03/wberm-mips-copy-300x245.jpg 300w" width="600"/></a> |
| | <figcaption class="wp-caption-text" id="caption-attachment-453"> |
| | <strong>Figure 3:</strong> Processing power available per dollar over time, measured in MIPS and 2007 US dollars. |
| | </figcaption> |
| | </figure> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <figure aria-describedby="caption-attachment-454" class="wp-caption alignnone" id="attachment_454" style="width: 600px"> |
| | <a href="http://aiimpacts.org/wp-content/uploads/2015/03/wberm-flops-copy.jpg"><img alt="Figure xxx: " class="wp-image-454" height="437" loading="lazy" sizes="(max-width: 600px) 100vw, 600px" src="https://aiimpacts.org/wp-content/uploads/2015/03/wberm-flops-copy-1024x746.jpg" srcset="https://aiimpacts.org/wp-content/uploads/2015/03/wberm-flops-copy-1024x746.jpg 1024w, https://aiimpacts.org/wp-content/uploads/2015/03/wberm-flops-copy-300x218.jpg 300w, https://aiimpacts.org/wp-content/uploads/2015/03/wberm-flops-copy.jpg 1030w" width="600"/></a> |
| | <figcaption class="wp-caption-text" id="caption-attachment-454"> |
| | <strong>Figure 4:</strong> Processing power available per dollar over time, measured in FLOPS using the LINPACK benchmark and in 2007 US dollars |
| | </figcaption> |
| | </figure> |
| | </HTML> |
| | |
| | |
| | ==== Rieber and Muehlhauser ==== |
| | |
| | |
| | <HTML> |
| | <p>Muehlhauser and Rieber (2014) <a href="https://intelligence.org/2014/05/12/exponential-and-non-exponential/#footnote_7_11027">extended</a> <a href="http://web.mit.edu/cmagee/www/documents/15-koh_magee-tfsc_functional_approach_studying_technological_progress_vol73p1061-1083_2006.pdf">Koh and Magee’s</a> data on MIPS available per dollar to 2014 (data [not currently] available <a href="https://docs.google.com/spreadsheets/d/1qPBpgqxHsqQgcLLXJ5H-4yto9SPQinR4H0f9p5Dh4g4/edit#gid=952780094">here</a>). Koh and Magee’s data largely comes from Moravec (like Nordhaus’ above), though they too extended it some. Muehlhauser and Rieber produced Figure 5.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>In this data, performance since 1940 appears to be growing by a factor of ten roughly every 5 years (14.2 orders of magnitude in 74 years). In the first fourteen years of this century, log(MIPS/$) grew from roughly -0.7 to 2.8, which corresponds to one order of magnitude every four years (or 77% growth per year).</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <figure aria-describedby="caption-attachment-450" class="wp-caption alignnone" id="attachment_450" style="width: 600px"> |
| | <a href="http://aiimpacts.org/wp-content/uploads/2015/03/image-5.png"><img alt="" class="wp-image-450" height="434" loading="lazy" sizes="(max-width: 600px) 100vw, 600px" src="https://aiimpacts.org/wp-content/uploads/2015/03/image-5.png" srcset="https://aiimpacts.org/wp-content/uploads/2015/03/image-5.png 710w, https://aiimpacts.org/wp-content/uploads/2015/03/image-5-300x217.png 300w" width="600"/></a> |
| | <figcaption class="wp-caption-text" id="caption-attachment-450"> |
| | <strong>Figure 5:</strong> Rieber and Muehlhauser’s MIPS/$ data (modified to fix typo). |
| | </figcaption> |
| | </figure> |
| | </HTML> |
| | |
| | |
| | ==== Wikipedia ==== |
| | |
| | |
| | <HTML> |
| | <p><a href="http://en.wikipedia.org/wiki/FLOPS#Hardware_costs">Wikipedia</a> has a small list of hardware configurations that authors claim produce gigaFLOPS efficiently, along with their prices at different times in recent history. Their data does not appear to cite other sources mentioned above.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p><a href="/doku.php?id=ai_timelines:wikipedia_history_of_gflops_costs">Here</a> is their table, as of March 2 2015. Figure 6 shows inflation adjusted costs of gigaFLOPS over time, taken from the table. The examples in the table were apparently selected as follows:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <blockquote> |
| | <p>The “cost per GFLOPS” is the cost for a set of hardware that would theoretically operate at one billion floating-point operations per second. During the era when no single computing platform was able to achieve one GFLOPS, this table lists the total cost for multiple instances of a fast computing platform which speed sums to one GFLOPS. Otherwise, the least expensive computing platform able to achieve one GFLOPS is listed.</p> |
| | </blockquote> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>We find this table dubious. It lacks many citations, and the citations it has frequently lack detail. For instance, the claims that the collections of hardware specified produce a GFLOPS are often unsubstantiated. We spent around thirty minutes trying to substantiate the 2015 figure, to no avail. The figure is more than an order of magnitude cheaper than <a href="/doku.php?id=ai_timelines:current_flops_prices" title="Current FLOPS prices">current FLOPS prices</a> we found.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>In this data, the price of a gigaFLOPS falls by an order of magnitude roughly every four years (14 orders of magnitude in 54 years is 3.9 years per order of magnitude). Since 1997, each order of magnitude only took three years (5.7 orders of magnitude in 18 years). Note that there is very little data before 1997.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <figure aria-describedby="caption-attachment-447" class="wp-caption alignnone" id="attachment_447" style="width: 600px"> |
| | <a href="http://aiimpacts.org/wp-content/uploads/2014/12/price-of-gflops.png"><img alt="Price of GFLOPS in different years, adjusted to 2013 US dollars." class="size-full wp-image-447" height="371" loading="lazy" sizes="(max-width: 600px) 100vw, 600px" src="https://aiimpacts.org/wp-content/uploads/2014/12/price-of-gflops.png" srcset="https://aiimpacts.org/wp-content/uploads/2014/12/price-of-gflops.png 600w, https://aiimpacts.org/wp-content/uploads/2014/12/price-of-gflops-300x186.png 300w" width="600"/></a> |
| | <figcaption class="wp-caption-text" id="caption-attachment-447"> |
| | <strong>Figure 6:</strong> Price of GFLOPS in different years according to Wikipedia, adjusted to 2013 US dollars. |
| | </figcaption> |
| | </figure> |
| | </HTML> |
| | |
| | |
| | ==== Short term trends ==== |
| | |
| | |
| | <HTML> |
| | <p><em>Main article:</em> <a href="/doku.php?id=ai_timelines:2017_trend_in_the_cost_of_computing"><em><strong>Recent trends in the cost of computing</strong></em></a></p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>The cheapest hardware prices (for single precision FLOPS/$) are on track to fall by around an order of magnitude every 10-16 years, based on data from around 2011-2017. There was no particular sign of slowing between 2011 and 2017.</p> |
| | </HTML> |
| | |
| | |
| | ===== Summary ===== |
| | |
| | |
| | <HTML> |
| | <p>We have looked at four efforts to measure long term hardware price performance trajectories. Two of them are based on Moravec’s earlier effort, while the other two appear to be more independent (though we suspect still draw on similar sources). Two investigations measured (G)FLOPS, two measured MIPS, and one measured MSOPS.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>Results seem fairly consistent in recent decades, and for MIPS/$ in the longer run. There is insufficient data on FLOPS in the long run to check consistency. All four estimates of growth later than the 1990s produce 3.5-4 years as the time for price performance to to grow an order of magnitude (we did not include an estimate for recent years from Sandberg and Bostrom’s FLOPS data, since they did not make one and it was not straightforward to make one ourselves).<span class="easy-footnote-margin-adjust" id="easy-footnote-1-448"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-1-448" title='This is consistent with Sandberg and Bostrom&#8217;s estimate of the relationship between FLOPS and MIPS: &#8216;Fitting a relationship suggests that FLOPS scales as MIPS to the power of 0.89, i.e. slightly slower than unity&#8217; (<a href="http://www.fhi.ox.ac.uk/brain-emulation-roadmap-report.pdf">p89</a>).'><sup>1</sup></a></span> Though note that these measures are from different spans within that period, and use different benchmarks (two were MIPS, one FLOPS, one MSOPS). Only Rieber and Muehlhauser and Wikipedia have data after 2002. Though they give similar recent growth figures, it is not clear how consistent they are: Rieber and Muehlhauser’s data appears to decline sharply in the last few years, and appears to only use CPUs, while the Wikipedia data is fairly even, and moves to GPUs in later years.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>If we take an MSOPS to be more or less equivalent to a MIPS (as Nordhaus claims), then growth in MIPS since the 1940s is fairly consistent across studies, gaining an order of magnitude roughly every 5 years (Nordhaus), 5 years (Rieber and Muehlhauser) or 5.6 years (Sandberg and Bostrom). Note that the former two draw on similar data.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>Our two estimates of long run growth in FLOPS/$ differ substantially: we have gained an order of magnitude either every 4 years or every 7.7 years. However the four year estimate comes from Wikipedia, which only has two entries prior to 1990, while Sandberg and Bostrom have on the order of hundreds of entries from that period. Thus we rely on Sanberg and Bostrom here, and estimate FLOPS grow by an order of magnitude every 7.7 years.</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <p>Prior to the 1940s, growth appears to be ambiguous and small. It looks like 2.4 orders of magnitude over forty eight years in Rieber and Muehlhauser’s figure, for an order of magnitude every 20 years. Nordhaus measures it as negative.</p> |
| | </HTML> |
| | |
| | |
| | ===== Further work ===== |
| | |
| | |
| | <HTML> |
| | <p>Further work on this subject might:</p> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <ul> |
| | <li><div class="li">Check Moravec’s data, as it appears to be widely cited and reused (perhaps just check consistency between the fraction of data from Moravec and that added later from another source in existing datasets).</div></li> |
| | <li><div class="li">Separate different types of computers (e.g. treat desktop CPUs, supercomputers, and GPUs separately)</div></li> |
| | <li><div class="li">Find other datasets and analyses</div></li> |
| | <li><div class="li">Combine all of the datasets into one</div></li> |
| | <li><div class="li">Produce more relevant data</div></li> |
| | <li><div class="li">Construct and measure a more relevant benchmark</div></li> |
| | </ul> |
| | </HTML> |
| | |
| | |
| | <HTML> |
| | <ol class="easy-footnotes-wrapper"> |
| | <li><div class="li"> |
| | <span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-1-448"></span>This is consistent with Sandberg and Bostrom’s estimate of the relationship between FLOPS and MIPS: ‘Fitting a relationship suggests that FLOPS scales as MIPS to the power of 0.89, i.e. slightly slower than unity’ (<a href="http://www.fhi.ox.ac.uk/brain-emulation-roadmap-report.pdf">p89</a>).<a class="easy-footnote-to-top" href="#easy-footnote-1-448"></a> |
| | </div></li> |
| | </ol> |
| | </HTML> |
| | |
| | |
| |
