AI Impacts Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== 2015 FLOPS prices ======

// Published 18 January, 2018 //

<HTML>
<p>In April 2015, the lowest GFLOPS prices we could find were approximately $3/GFLOPS. However recent records of hardware performance from 2015 and earlier imply substantially lower prices, suggesting that something confusing has happened with these sources of data. We have not resolved this.</p>
</HTML>



===== Recent data =====


<HTML>
<p>We have not finished exploring the apparent discrepancies between 2015 prices for performance and current records of 2015 prices for performance. However in the data described in our <a href="/doku.php?id=ai_timelines:2017_trend_in_the_cost_of_computing">2017 assessment</a> of recent price trends (key figure <a href="http://aiimpacts.org/wp-content/uploads/2017/10/chart-43.png">here</a>), prices appear to have been below $1 since 2008.<span class="easy-footnote-margin-adjust" id="easy-footnote-1-1035"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-1-1035" title='Other examples of apparently very cheap hardware from 2015 or earlier can be seen at the bottom of the sheet called &amp;#8216;Oct 2017 Update &amp;#8211; misc &amp;#8211; incomplete, misleading&amp;#8217; in &lt;a href="https://docs.google.com/spreadsheets/d/1yqX2cENwkOxC26wV_sBOvV0NxHzzfmL6tU7StzrFXRc/edit?usp=sharing"&gt;this spreadsheet&lt;/a&gt;.'><sup>1</sup></a></span> The measurements are not entirely comparable, but we would not expect the differences to produce such a large price difference.</p>
</HTML>


===== 2015 research =====


<HTML>
<p><em>The rest of this page is largely taken from our page written in 2015.</em></p>
</HTML>


<HTML>
<p>In April 2015, the lowest recorded GFLOPS prices we knew of were approximately $3/GFLOPS, for various CPU and GPU combinations. Amortized over three years, this was $1.1E-13/FLOPShour. Prices in the $3-5/GFLOPS range seemed to be common, for GPU and CPU combinations and sometimes for supercomputers. Using CPUs, prices were at least $11/GFLOPS, and computing as a service cost more like $160/GFLOPS.</p>
</HTML>


==== Background ====


<HTML>
<p>We have written about <a href="/doku.php?id=ai_timelines:trends_in_the_cost_of_computing" title="Trends in the cost of computing">long term trends</a> in the costs of computing hardware. We were interested in evaluating the current prices more thoroughly, both to validate the long term trend data, and because current hardware prices are particularly important to know about.</p>
</HTML>


==== Details ====


<HTML>
<p>We separately investigated CPUs, GPUs, computing as a service, and supercomputers. In all categories, we collected some contemporary instances which we judged heuristically as especially likely to be cost-effective. We did not find any definitive source on the most cost-effective in any category, or in general, so our examples are probably not the very cheapest.  Nevertheless, these figures give a crude sense for the cost of computation in the contemporary market. Our full dataset of CPUs, GPUs and supercomputers is <a href="https://docs.google.com/spreadsheets/d/1yqX2cENwkOxC26wV_sBOvV0NxHzzfmL6tU7StzrFXRc/edit?usp=sharing">here</a>, and contains data on twenty two machines. Our data on computing as a service is all included in this page.</p>
</HTML>


=== Included costs ===


<HTML>
<p>For CPUs and GPUs, we list the price of the CPU and/or GPU (GPUs were always used with a CPU, so we include the cost for both), but not other computer components. We compared prices between <a href="https://drive.google.com/file/d/1xTA4LoooCuHhCWLhkJZ2ZbOWDxso3a0UILO2JxLn5CweadJVJUziux7CMiumtITXTOjXfttY5zNHzCee/view?usp=sharing">one complete rack server</a> and the set of four <a href="http://www.ebay.com/itm/like/351337480917?lpid=82&amp;chn=ps">processors</a> inside it, and found the complete server was around 36% more expensive ($30,000 vs. $22,000). We expect this is representative at this scale, but diminishes with scale.</p>
</HTML>


<HTML>
<p>For computing services, we list the cheapest price for renting the instance for a long period, with no additional features. We do not include spot prices.</p>
</HTML>


<HTML>
<p>For supercomputers, we list costs cited, which don’t tend to come with elaboration. We expect that they only include upfront costs, and that most of the costs are for hardware.</p>
</HTML>


<HTML>
<p>We have not included the costs of energy or other ongoing expenses in any prices. Non-energy costs are hard to find, and we suspect a relatively small and consistent fraction of costs. Energy costs appear to be around 10% of hardware costs. For instance, the Intel Xeon E5-2699 uses 527.8 watts and costs $5,190.<span class="easy-footnote-margin-adjust" id="easy-footnote-2-1035"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-2-1035" title='The processor can be bought &lt;a href="http://www.serversupply.com/products/part_search/pid_lookup.asp?pid=229567&amp;amp;gclid=Cj0KEQjwi-moBRDL4Omf9d_LndMBEiQAQtFf83x42USI1XM-_KXMDkkDbi8NyYzbLFemeykjBuQfPrYaAlew8P8HAQ"&gt;here&lt;/a&gt; for $5,190 as of April 1 2015. Its energy consumption is &lt;a href="http://www.tomshardware.com/reviews/intel-xeon-e5-2600-v3-haswell-ep,3932-9.html"&gt;527.8 watts&lt;/a&gt; under load, or 90.9 watts idle.'><sup>2</sup></a></span> Over three years, with $0.05/kWh this is $694, or 13% of the hardware cost. Titan also uses 13% of its hardware costs in energy over three years.<span class="easy-footnote-margin-adjust" id="easy-footnote-3-1035"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-3-1035" title='Titan &lt;a href="http://en.wikipedia.org/wiki/Titan_(supercomputer)"&gt;cost&lt;/a&gt; about $4000 dollars per hour amortized over 3 years, and consumes about 10M watts, at a cost of $500 per hour (assuming $0.05 per kWh), which is also 13% of its hardware cost.'><sup>3</sup></a></span> We might add these costs later for a more precise estimate.</p>
</HTML>


=== FLOPS measurements ===


<HTML>
<p>To our knowledge we report only empirical performance figures from benchmark tests, rather than theoretical maximums. We sometimes use figures for <a href="http://en.wikipedia.org/wiki/LINPACK_benchmarks">LINPACK</a> and sometimes for <a href="http://matthewrocklin.com/blog/work/2012/10/29/Matrix-Computations/">DGEMM</a> benchmarks, depending on which are available. <a href="http://browser.primatelabs.com/">Geekbench</a> in particular does not use the common LINPACK, but <a href="http://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms">LINPACK relies heavily on DGEMM</a>, suggesting DGEMM is fairly comparable. We guess they differ by around 10%.<span class="easy-footnote-margin-adjust" id="easy-footnote-4-1035"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-4-1035" title='&lt;a href="http://www.nvidia.com/content/gtc-2010/pdfs/2057_gtc2010.pdf"&gt;This presentation&lt;/a&gt; (page &amp;#8216;Results on a single node&amp;#8217;) reports Linpack performance of 95% and 89% of DGEMM performance for their hardware in two tests.'><sup>4</sup></a></span></p>
</HTML>


==== Prices ====


=== Central processing units (CPUs) ===


<HTML>
<p>We found prices and performance data for five contemporary CPUs, including three different instances of one of them. They ranged from $11-354/GFLOPS with most prices below $100/GFLOPS.<span class="easy-footnote-margin-adjust" id="easy-footnote-5-1035"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-5-1035" title='Muehlhauser and Rieber &lt;a href="https://intelligence.org/2014/05/12/exponential-and-non-exponential/#footnote_7_11027"&gt;extended&lt;/a&gt; &lt;a href="http://web.mit.edu/cmagee/www/documents/15-koh_magee-tfsc_functional_approach_studying_technological_progress_vol73p1061-1083_2006.pdf"&gt;Koh and Magee&amp;#8217;s&lt;/a&gt; data on MIPS available per dollar to 2014 (data available &lt;a href="https://docs.google.com/spreadsheets/d/1qPBpgqxHsqQgcLLXJ5H-4yto9SPQinR4H0f9p5Dh4g4/edit#gid=952780094"&gt;here&lt;/a&gt;). Their 2014 datapoint is for Intel Core i5-4430 and is 607 MIPS/$, and is roughly in line with their figures for recent years. According to &lt;a href="http://browser.primatelabs.com/geekbench3/2226239"&gt;Geekbench&lt;/a&gt;, it achieves 15.7 GFLOPS on the DGEMM benchmark. According to PCworld, it initially cost &lt;a href="http://www.superbiiz.com/detail.php?p=I5_4430&amp;amp;c=fr&amp;amp;pid=767700e77d6b13eb4f36ade3ae3993be712499a357b94ea45d012390fed8c810&amp;amp;gclid=CjwKEAjwru6oBRDDp4jRj4bL_xASJADJ2obyp2Djq4iZvzJrOMH6sArIFrcFpoOh-Mi7uih_rS-KlhoCGmDw_wcB"&gt;$175&lt;/a&gt;, and we have not found cheaper prices than this. This implies $11.14/GFLOPS. However Muehlhauser and Rieber seem to report a price of $48, which would make it $3.06/GFLOPS, highly competitive with GPUs and supercomputers. They also cite CPUworld, so we suspect this is an error. Either way, this CPU would not be substantially cheaper than the best GPUs, so does not alter our results.'><sup>5</sup></a></span> The cheapest of these CPUs still looks several times more expensive than some GPUs and supercomputers, so we did not investigate these numbers in great depth, or search far for cheaper CPUs.</p>
</HTML>


=== Graphics processing units (GPUs) ===


<HTML>
<p>We found performance data for six recent combinations of CPUs and GPUs (with much overlap between CPUs and GPUs between combinations. They ranged from $3.22/GFLOPS to $4.17/GFLOPS.</p>
</HTML>


<HTML>
<p>Note that graphics cards are typically significantly restricted in the kinds of applications they can run efficiently; this performance is achieved for highly regular computations that can be carried out in parallel throughout a GPU (of the sort that are required for rendering scenes, but which have also proved useful in scientific computing).</p>
</HTML>


=== Computing as service ===


<HTML>
<p>Another way to purchase FLOPS is via virtual computers.</p>
</HTML>


<HTML>
<p>Amazon <a href="http://en.wikipedia.org/wiki/Amazon_Elastic_Compute_Cloud">Elastic Cloud Compute</a> (EC2) is a major seller of virtual computing. Based on their <a href="https://aws.amazon.com/ec2/pricing/" rel="nofollow">current pricing</a>, renting a <a href="https://aws.amazon.com/ec2/instance-types/">c4.8xlarge</a> instance costs about $1.17 / hour.<span class="easy-footnote-margin-adjust" id="easy-footnote-6-1035"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-6-1035" title="The effective hourly rate, if you purchase 3 years of computing, and pay upfront, is $1.1653 per hour."><sup>6</sup></a></span> This is their largest instance optimized for computing performance (rather than e.g. memory). A c4.8xlarge instance <a href="http://browser.primatelabs.com/geekbench3/1694602">delivers</a> around 97.5 GFLOPS.<span class="easy-footnote-margin-adjust" id="easy-footnote-7-1035"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-7-1035" title=' &lt;a href="http://browser.primatelabs.com/"&gt;Geekbench Browser&lt;/a&gt; allows users to measure performance in FLOPS using a variety of tasks. 97.5 is the multi-core DGEMM score &lt;a href="http://browser.primatelabs.com/geekbench3/1694602"&gt;a user reported&lt;/a&gt; for c4.8xlarge. We use a multi-core score because the cost cited is for purchasing all of the cores. On other tasks, Geekbench &lt;a href="http://browser.primatelabs.com/geekbench3/1694602"&gt;reports&lt;/a&gt; scores from 46 to 199 GFLOPS. We do not know how reliable Geekbench reports are.'><sup>7</sup></a></span> This implies that a GFLOPShour costs $0.012. If we suppose this is an alternative to buying computer hardware, then the relevant time horizon is about three years. Over three years, renting this hardware will cost $316/GFLOPS, i.e. around two orders of magnitude more than buying GFLOPS in the form of GPUs.</p>
</HTML>


<HTML>
<p>Other sources of virtual computing seem to be similarly priced. An <a href="http://www.infoworld.com/d/cloud-computing/ultimate-cloud-speed-tests-amazon-vs-google-vs-windows-azure-237169?page=0,2">informal comparison</a> of computing providers suggests that on a set of “real-world java benchmarks” three providers are quite closely comparable, with all between just above Amazon’s price and just under half Amazon’s price for completing the benchmarks, across different instance sizes. This analysis also suggests Amazon is a relatively costly provider, and suggests a cheap price for virtual computing is closer to $0.006/GFLOPShour or $160/GFLOPS over three years.</p>
</HTML>


<HTML>
<p>Even with this optimistic estimate, virtual computing appears to cost something like fifty times more than GPUs. This high price is presumably partly because there are non-hardware costs which we have not accounted for in the prices of buying hardware, but are naturally included in the cost of renting it. However it is unlikely that these additional costs make up a factor of fifty.</p>
</HTML>


=== Supercomputing ===


<HTML>
<p>The Titan supercomputer <a href="http://en.wikipedia.org/wiki/Titan_(supercomputer)">purportedly</a> cost about $97M to produce, or about $4,000 dollars per hour amortized over 3 years. It performs 17,590,000 GFLOPS which comes to $5.51/GFLOPS. This makes it around the same price as the cheapest GPUs. It is made of a combination of GPUs and CPUs, so this similarity is unsurprising.</p>
</HTML>


<HTML>
<p>The other six built supercomputers we looked at were more expensive, ranging up to $95/GFLOPS. Another cost-effective supercomputer, the L-CSC, was being built at the time it was most recently reported on, and while it should be completed now we could not find more data on it. Extrapolating from the figures before it was finished, when completed it should cost $2.39/GFLOPS, and thus be the cheapest source of FLOPS we are aware of.</p>
</HTML>


==== Summary ====


<HTML>
<p>The lowest recorded GFLOPS prices we know of are approximately $3/GFLOPS, for various CPU and GPU combinations. Amortized over three years, this is $1.1E-13/FLOPShour. Prices in the $3-5/GFLOPS range seem to be common, for GPU and CPU combinations and sometimes for supercomputers. Using CPUs, prices are at least $11/GFLOPS, and computing as a service costs more like $160/GFLOPS.</p>
</HTML>


<HTML>
<ol class="easy-footnotes-wrapper">
<li><div class="li">
<span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-1-1035"></span>Other examples of apparently very cheap hardware from 2015 or earlier can be seen at the bottom of the sheet called ‘Oct 2017 Update – misc – incomplete, misleading’ in <a href="https://docs.google.com/spreadsheets/d/1yqX2cENwkOxC26wV_sBOvV0NxHzzfmL6tU7StzrFXRc/edit?usp=sharing">this spreadsheet</a>.<a class="easy-footnote-to-top" href="#easy-footnote-1-1035"></a>
</div></li>
<li><div class="li">
<span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-2-1035"></span>The processor can be bought <a href="http://www.serversupply.com/products/part_search/pid_lookup.asp?pid=229567&amp;gclid=Cj0KEQjwi-moBRDL4Omf9d_LndMBEiQAQtFf83x42USI1XM-_KXMDkkDbi8NyYzbLFemeykjBuQfPrYaAlew8P8HAQ">here</a> for $5,190 as of April 1 2015. Its energy consumption is <a href="http://www.tomshardware.com/reviews/intel-xeon-e5-2600-v3-haswell-ep,3932-9.html">527.8 watts</a> under load, or 90.9 watts idle.<a class="easy-footnote-to-top" href="#easy-footnote-2-1035"></a>
</div></li>
<li><div class="li">
<span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-3-1035"></span>Titan <a href="http://en.wikipedia.org/wiki/Titan_(supercomputer)">cost</a> about $4000 dollars per hour amortized over 3 years, and consumes about 10M watts, at a cost of $500 per hour (assuming $0.05 per kWh), which is also 13% of its hardware cost.<a class="easy-footnote-to-top" href="#easy-footnote-3-1035"></a>
</div></li>
<li><div class="li">
<span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-4-1035"></span><a href="http://www.nvidia.com/content/gtc-2010/pdfs/2057_gtc2010.pdf">This presentation</a> (page ‘Results on a single node’) reports Linpack performance of 95% and 89% of DGEMM performance for their hardware in two tests.<a class="easy-footnote-to-top" href="#easy-footnote-4-1035"></a>
</div></li>
<li><div class="li">
<span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-5-1035"></span>Muehlhauser and Rieber <a href="https://intelligence.org/2014/05/12/exponential-and-non-exponential/#footnote_7_11027">extended</a> <a href="http://web.mit.edu/cmagee/www/documents/15-koh_magee-tfsc_functional_approach_studying_technological_progress_vol73p1061-1083_2006.pdf">Koh and Magee’s</a> data on MIPS available per dollar to 2014 (data available <a href="https://docs.google.com/spreadsheets/d/1qPBpgqxHsqQgcLLXJ5H-4yto9SPQinR4H0f9p5Dh4g4/edit#gid=952780094">here</a>). Their 2014 datapoint is for Intel Core i5-4430 and is 607 MIPS/$, and is roughly in line with their figures for recent years. According to <a href="http://browser.primatelabs.com/geekbench3/2226239">Geekbench</a>, it achieves 15.7 GFLOPS on the DGEMM benchmark. According to PCworld, it initially cost <a href="http://www.superbiiz.com/detail.php?p=I5_4430&amp;c=fr&amp;pid=767700e77d6b13eb4f36ade3ae3993be712499a357b94ea45d012390fed8c810&amp;gclid=CjwKEAjwru6oBRDDp4jRj4bL_xASJADJ2obyp2Djq4iZvzJrOMH6sArIFrcFpoOh-Mi7uih_rS-KlhoCGmDw_wcB">$175</a>, and we have not found cheaper prices than this. This implies $11.14/GFLOPS. However Muehlhauser and Rieber seem to report a price of $48, which would make it $3.06/GFLOPS, highly competitive with GPUs and supercomputers. They also cite CPUworld, so we suspect this is an error. Either way, this CPU would not be substantially cheaper than the best GPUs, so does not alter our results.<a class="easy-footnote-to-top" href="#easy-footnote-5-1035"></a>
</div></li>
<li><div class="li">
<span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-6-1035"></span>The effective hourly rate, if you purchase 3 years of computing, and pay upfront, is $1.1653 per hour.<a class="easy-footnote-to-top" href="#easy-footnote-6-1035"></a>
</div></li>
<li><div class="li">
<span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-7-1035"></span> <a href="http://browser.primatelabs.com/">Geekbench Browser</a> allows users to measure performance in FLOPS using a variety of tasks. 97.5 is the multi-core DGEMM score <a href="http://browser.primatelabs.com/geekbench3/1694602">a user reported</a> for c4.8xlarge. We use a multi-core score because the cost cited is for purchasing all of the cores. On other tasks, Geekbench <a href="http://browser.primatelabs.com/geekbench3/1694602">reports</a> scores from 46 to 199 GFLOPS. We do not know how reliable Geekbench reports are.<a class="easy-footnote-to-top" href="#easy-footnote-7-1035"></a>
</div></li>
</ol>
</HTML>
AI Impacts Wiki

User Tools

Site Tools

Page Tools
