AI Impacts Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Global computing capacity ======

// Published 16 February, 2016; last updated 17 April, 2020 //

//This page is out of date and its contents may have been inaccurate in 2015, in light of new information that we are yet to integrate. See [[ai_timelines:hardware_and_ai_timelines:computing_capacity_of_all_gpus_and_tpus|Computing capacity of all GPUs and TPUs]] for a related and more up-to-date analysis.//


<HTML>
<p>Computing capacity worldwide was probably around 2 x 10<sup>20 </sup>– 1.5 x 10<sup>21</sup> FLOPS, at around the end of 2015.</p>
</HTML>



===== Support =====


<HTML>
<p>We are not aware of recent, plausible estimates for hardware capacity.</p>
</HTML>


<HTML>
<p>Vipul Naik estimated global hardware capacity in February 2014, based on Hilbert &amp; Lopez’s estimates for 1986-2007. He calculated that if all computers ran at full capacity, they would perform 10-1000 zettaFLOPS, i.e. 10<sup>22 </sup>– 10<sup>24</sup> FLOPS.<span class="easy-footnote-margin-adjust" id="easy-footnote-1-644"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-1-644" title='&amp;#8220;My current guesstimate for total computation being done would be 0.1-10 zettaFLOPS and current estimate for how much computation can be done if all computers ran at full capacity would be 10-1000 zettaFLOPS (but this would entail prohibitive energy costs and not be sustainable).&amp;#8221;&lt;/p&gt; &lt;p&gt;&amp;#8211; Vipul Naik, &lt;a href="https://intelligence.org/wp-content/uploads/2014/02/Naik-Distribution-of-Computation.pdf"&gt;research notes&lt;/a&gt;'><sup>1</sup></a></span> We think these are substantial overestimates, because producing so much computing hardware would cost more than 10% of gross world product (GWP), which is implausibly high. The most cost-efficient computing hardware we are aware of today are GPUs, which still cost about <a href="/doku.php?id=ai_timelines:current_flops_prices">$3/GFLOPS</a>, or $1/GFLOPSyear if we assume hardware is used for around three years. This means maintaining hardware capable of 10<sup>22 </sup>– 10<sup>24</sup> FLOPS would cost at least $10<sup>13 </sup>– $10<sup>15 </sup> per year. Yet <a href="https://en.wikipedia.org/wiki/Gross_world_product">gross world product</a> (GWP) is only around $8 x 10<sup>13</sup>, so this would imply hardware spending constitutes more than 13% – 1300% of GWP. Even the lower end of this range seems implausible.<span class="easy-footnote-margin-adjust" id="easy-footnote-2-644"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-2-644" title='One might wonder if the total stock of hardware seems high because hardware lasts for much longer than three years. This probably does not account for it, because the growth rate is high enough that even if substantially older hardware persisted, it would make up a small share of total hardware. For instance in 2007 the world&amp;#8217;s computing capacity (for the general purpose and application specific computers &lt;a href="http://ijoc.org/index.php/ijoc/article/view/1562/742"&gt;Hilbert &amp;amp; Lopez&lt;/a&gt; estimated) was only 2 x 10&lt;sup&gt;20 &lt;/sup&gt;IPS (similar to FLOPS), roughly 0.02%-2% of Naik&amp;#8217;s estimate.&lt;/p&gt; '><sup>2</sup></a></span></p>
</HTML>


<HTML>
<p>One way to estimate global hardware capacity ourselves is based on annual hardware spending. This is slightly complicated because hardware lasts for several years. So to calculate how much hardware exists in 2016, we would ideally like to know how much was bought in every preceding year, and also how much of each annual hardware purchase has already been discarded. To simplify matters, we will instead assume that hardware lasts for around three years.</p>
</HTML>


<HTML>
<p>It appears that very roughly $300bn-$1,500bn was spent on hardware in 2015.<span class="easy-footnote-margin-adjust" id="easy-footnote-3-644"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-3-644" title='&amp;#8220;In 2012, the worldwide computing hardware spending is expected at 418 billion U.S. dollars.&amp;#8221; &amp;#8211; &lt;a href="http://www.statista.com/statistics/203946/global-computing-hardware-spending-forecast/"&gt;Statista&lt;/a&gt;&lt;/p&gt; &lt;p&gt;&lt;a href="http://www.statista.com/statistics/422802/hardware-spending-forecast-worldwide/"&gt;Statista&amp;#8217;s&lt;/a&gt; figure of &amp;#8216;Forecast hardware spendings worldwide from 2013 to 2019 (in billion U.S. dollars)&amp;#8217; reports a 2013 figure of $987bn, increasing to $1075bn in 2015. It is unclear why these spending forecasts differ so much from Statista&amp;#8217;s reported 2012 spending.&lt;/p&gt; &lt;p&gt;&lt;a href="http://www.statista.com/statistics/268633/worldwide-revenue-from-computer-hardware-since-2005/"&gt;Statista&lt;/a&gt; also reports a prediction of 2016 hardware revenue at $409bn Euro, which is around $447bn USD. It looks like the prediction was made in 2012. Note that revenue is not identical to spending, but is probably a reasonable proxy.&lt;/p&gt; &lt;p&gt;For 2009, Reuters reports a substantially lower revenue figure than Statista, suggesting Statista figures may be systematically high, e.g. by being relatively inclusive:&lt;/p&gt; &lt;p&gt;&amp;#8220;The global computer hardware market had total revenue of $193.2 billion in 2009, representing a compound annual growth rate (CAGR) of 5.4% for the period spanning 2005-2009.&amp;#8221; &amp;#8211; &lt;a href="http://www.reuters.com/article/idUS230848+13-Jul-2011+BW20110713"&gt;Research and Markets press release, Reuters&lt;/a&gt;,&lt;/p&gt; &lt;p&gt;&lt;a href="http://www.statista.com/statistics/268633/worldwide-revenue-from-computer-hardware-since-2005/"&gt;Statista&lt;/a&gt;&amp;#8216;s figure indicates revenue of 296 billion Euros, or around $320 billion USD in 2009 (this is the same figure as for 2007, which may be the only number you can see without a subscription—so while it may look like we made an error here, we do have the figure for the correct year). This is around 50% more than the Research and Markets press release.&lt;/p&gt; &lt;p&gt;From these figures we estimate that spending on hardware in 2015 was $300bn-$1,500bn'><sup>3</sup></a></span> We <a href="/doku.php?id=ai_timelines:current_flops_prices">previously estimated</a> that the cheapest available hardware (in April 2015) was around $3/GFLOPS. So if humanity spent $300bn-$1,500bn on hardware in 2015, and it was mostly the cheapest hardware, then the hardware we bought should perform around 10<sup>20</sup> – 5 x 10<sup>20</sup> FLOPS. If we multiply this by three to account for the previous two years’ hardware purchases still being around, we have about  3 x 10<sup>20 </sup>– 1.5 x 10<sup>21</sup> FLOPS.</p>
</HTML>


<HTML>
<p>This estimate is rough, and could be improved in several ways. Most likely, more hardware is being bought each year than the previous year. So approximating last years’ hardware purchase to this years’ will yield too much hardware. In particular, the faster global hardware is growing, the closer the total is to whatever humanity bought this year (that is, counterintuitively, if you think hardware is growing faster, you should suppose that there is <em>less of it</em> by this particular method of estimation). Furthermore, perhaps a lot of hardware is not the cheapest for various reasons. This too suggests there is less hardware than we estimated.</p>
</HTML>


<HTML>
<p>On the other hand, hardware may often last for more than three years (we don’t have a strong basis for our assumption there). And our prices are from early 2015, so hardware is likely somewhat cheaper now (in early 2016). Our guess is that overall these considerations mean our estimate should be lower, but probably by less than a factor of four in total. This suggests 7.5 x 10<sup>19</sup> – 1.5 x 10<sup>21</sup> FLOPS of hardware.</p>
</HTML>


<HTML>
<p>However <a href="http://ijoc.org/index.php/ijoc/article/view/1562/742">Hilbert &amp; Lopez</a> (2012) estimated that in 2007 the world’s computing capacity was around 2 x 10<sup>20 </sup>IPS (similar to FLOPS) already, after constructing a detailed inventory of technologies.<span class="easy-footnote-margin-adjust" id="easy-footnote-4-644"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-4-644" title='&amp;#8220;It is written from the perspective of the results of our recent inventory of 60 technological categories between 1986 and 2007 (measured in bits and MIPS [million-instructions-per-second])&amp;#8230;&amp;#8221;&lt;/p&gt; &lt;div class="page" title="Page 1"&gt; &lt;div class="layoutArea"&gt; &lt;div class="column"&gt; &lt;p&gt;&amp;#8220;&amp;#8230;The combined capacity of both groups of computers has grown from 730 tera-IPS in 1986 (73010^12 instructions per second), over 22.5 peta-IPS in 1993 (22.510^15 instructions per second), to 1.8 exa-IPS in 2000 (1.810^18 IPS), and 196 exa-IPS in 2007 (or roughly 210^20 instructions per second).&amp;#8221;&lt;/p&gt; &lt;/div&gt; &lt;/div&gt; &lt;/div&gt; &lt;p&gt;&amp;#8211;  &lt;a href="http://ijoc.org/index.php/ijoc/article/view/1562/742"&gt;Hilbert &amp;amp; Lopez&lt;/a&gt; (2012)&lt;/p&gt; '><sup>4</sup></a></span> Their estimate does not appear to conflict with data about the global economy at the time.<span class="easy-footnote-margin-adjust" id="easy-footnote-5-644"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-5-644" title='In 2007, GWP &lt;a href="https://web.archive.org/web/20080401214624/https://www.cia.gov/library/publications/the-world-factbook/fields/2001.html"&gt;was probably&lt;/a&gt; about $66T (in 2007 dollars). According to &lt;a href="http://ijoc.org/index.php/ijoc/article/view/1562/742"&gt;Hilbert &amp;amp; Lopez&lt;/a&gt;, the world could then perform 2 x 10&lt;sup&gt;20&lt;/sup&gt; IPS, which is  2 x 10&lt;sup&gt;14&lt;/sup&gt; MIPS. According to &lt;a href="https://intelligence.org/2014/05/12/exponential-and-non-exponential/#footnote_7_11027"&gt;Muehlhauser &amp;amp; Rieber&lt;/a&gt;, hardware cost roughly $5 x 10&lt;sup&gt;-3&lt;/sup&gt;/MIPS in 2007. Thus the total value of hardware would have been around $5 x 10&lt;sup&gt;-3&lt;/sup&gt;/MIPS x 2 x 10&lt;sup&gt;14&lt;/sup&gt; MIPS = $10&lt;sup&gt;12&lt;/sup&gt; (a trillion dollars), or 1.5% of GWP.'><sup>5</sup></a></span> Growth is unlikely to have been negative since 2007, though Hilbert &amp; Lopez may have overestimated. So we revise our estimate to 2 x 10<sup>20</sup> – 1.5 x 10<sup>21</sup> FLOPS for the end of 2015.</p>
</HTML>


<HTML>
<p>This still suggests that in the last nine years, the world’s hardware has grown by a factor of 1-7.5, implying a growth rate of 0%-25%. Even 25% would be quite low compared to growth rates between 1986 and 2007 according to <a href="http://ijoc.org/index.php/ijoc/article/view/1562/742">Hilbert &amp; Lopez</a> (2012), which were 61% for general purpose computing and 86% for the set of ASICs they studied (which in 2007 accounted for 32 times as much computing as general purpose computers).<span class="easy-footnote-margin-adjust" id="easy-footnote-6-644"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-6-644" title='&amp;#8220;The respective compound annual growth rates between 1986–2007 were 61% for general-purpose computations and 86% for application-specific computations, which is 10 and 14 times faster than global GDP during that period, respectively.&amp;#8221;&lt;/p&gt; &lt;p&gt;&lt;a href="http://ijoc.org/index.php/ijoc/article/view/1562/742"&gt;Hilbert &amp;amp; Lopez&lt;/a&gt; (2012)&lt;/p&gt; '><sup>6</sup></a></span> However if we are to distrust estimates which imply hardware is a large fraction of GWP, then we must expect hardware growth has slowed substantially in recent years. For comparison, our estimates are around 2-15% of Naik’s lower bound, and suggest that hardware constitutes around 0.3%-1.9% of GWP.</p>
</HTML>


<HTML>
<p>Such large changes in the long run growth rate are surprising to us, and—if they are real—we are unsure what produced them. One possibility is that hardware prices have stopped falling so fast (i.e. Moore’s Law is ending for the price of computation). Another is that spending on hardware decreased for some reason, for instance because people stopped enjoying large returns from additional hardware. We think this question deserves further research.</p>
</HTML>


===== Implications =====


==== Global computing capacity in terms of human brains ====


<HTML>
<p>According to <a href="/doku.php?id=ai_timelines:brain_performance_in_flops">different estimates</a>, the human brain performs the equivalent of between 3 x 10<sup>13</sup> and 10<sup>25</sup> FLOPS. The median estimate we know of is 10<sup>18</sup> FLOPS. According to that median estimate and our estimate of global computing hardware, if the world’s entire computing capacity could be directed at running minds around as efficient as those of humans, we would have the equivalent of 200-1500 extra human minds.<span class="easy-footnote-margin-adjust" id="easy-footnote-7-644"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-7-644" title="2 x 10&lt;sup&gt;20 &lt;/sup&gt;/10&lt;sup&gt;18 &lt;/sup&gt; = 2 x 10&lt;sup&gt;2&lt;/sup&gt;&lt;/p&gt; &lt;p&gt;1.5 x 10&lt;sup&gt;21&lt;/sup&gt;/10&lt;sup&gt;18&lt;/sup&gt;=1.5 x 10&lt;sup&gt;3&lt;/sup&gt;"><sup>7</sup></a></span> That is, turning all of the world’s hardware into human-efficiency minds at present would increase the world’s population of minds by at most about 0.00002%. If we select the most favorable set of estimates for producing large numbers, turning all of the world’s computing hardware into minds as efficient as humans’ would produce around 50 million extra minds, increasing the world’s effective population by about 1%.<span class="easy-footnote-margin-adjust" id="easy-footnote-8-644"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-8-644" title="1.5 x 10&lt;sup&gt;21&lt;/sup&gt; FLOPS of hardware divided by 3 x 10&lt;sup&gt;13&lt;/sup&gt; FLOPS/brain gives us 5 x 10&lt;sup&gt;7 &lt;/sup&gt;minds."><sup>8</sup></a></span></p>
</HTML>


<HTML>
<figure aria-describedby="caption-attachment-713" class="wp-caption alignnone" id="attachment_713" style="width: 600px">
<img alt="Figure: Projected number of human brains equivalent " class="wp-image-713" height="404" sizes="(max-width: 600px) 100vw, 600px" src="https://aiimpacts.org/wp-content/uploads/2016/02/Global-hardware-in-FLOPS-equivalent-brains.png" srcset="https://aiimpacts.org/wp-content/uploads/2016/02/Global-hardware-in-FLOPS-equivalent-brains.png 1667w, https://aiimpacts.org/wp-content/uploads/2016/02/Global-hardware-in-FLOPS-equivalent-brains-300x202.png 300w, https://aiimpacts.org/wp-content/uploads/2016/02/Global-hardware-in-FLOPS-equivalent-brains-768x517.png 768w, https://aiimpacts.org/wp-content/uploads/2016/02/Global-hardware-in-FLOPS-equivalent-brains-1024x689.png 1024w" width="600"/>
<figcaption class="wp-caption-text" id="caption-attachment-713">
                  Figure: Projected number of human brains equivalent to global hardware under various assumptions. For brains, ‘small’ = 3 x 10^ 13, ‘median’ = 10^18, ‘large’ = 10^25. For ‘world hardware’, ‘high’ =2 x 10^20, ‘low’ = 1.5 x 10^21. ‘Growth’ is growth in computing hardware, the unlabeled default used in most projections is 25% per annum (our estimate above), ‘high’ = 86% per annum (the apparent growth rate in ASIC hardware in around 2007).
                </figcaption>
</figure>
</HTML>




<HTML>
<ol class="easy-footnotes-wrapper">
<li><div class="li">
<span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-1-644"></span>“My current guesstimate for total computation being done would be 0.1-10 zettaFLOPS and current estimate for how much computation can be done if all computers ran at full capacity would be 10-1000 zettaFLOPS (but this would entail prohibitive energy costs and not be sustainable).”
                  <p>– Vipul Naik, <a href="https://intelligence.org/wp-content/uploads/2014/02/Naik-Distribution-of-Computation.pdf">research notes</a><a class="easy-footnote-to-top" href="#easy-footnote-1-644"></a></p>
</div></li>
<li><div class="li">
<span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-2-644"></span>One might wonder if the total stock of hardware seems high because hardware lasts for much longer than three years. This probably does not account for it, because the growth rate is high enough that even if substantially older hardware persisted, it would make up a small share of total hardware. For instance in 2007 the world’s computing capacity (for the general purpose and application specific computers <a href="http://ijoc.org/index.php/ijoc/article/view/1562/742">Hilbert &amp; Lopez</a> estimated) was only 2 x 10<sup>20 </sup>IPS (similar to FLOPS), roughly 0.02%-2% of Naik’s estimate.<a class="easy-footnote-to-top" href="#easy-footnote-2-644"></a>
</div></li>
<li><div class="li">
<span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-3-644"></span>“In 2012, the worldwide computing hardware spending is expected at 418 billion U.S. dollars.” – <a href="http://www.statista.com/statistics/203946/global-computing-hardware-spending-forecast/">Statista</a>
<p><a href="http://www.statista.com/statistics/422802/hardware-spending-forecast-worldwide/">Statista’s</a> figure of ‘Forecast hardware spendings worldwide from 2013 to 2019 (in billion U.S. dollars)’ reports a 2013 figure of $987bn, increasing to $1075bn in 2015. It is unclear why these spending forecasts differ so much from Statista’s reported 2012 spending.</p>
<p><a href="http://www.statista.com/statistics/268633/worldwide-revenue-from-computer-hardware-since-2005/">Statista</a> also reports a prediction of 2016 hardware revenue at $409bn Euro, which is around $447bn USD. It looks like the prediction was made in 2012. Note that revenue is not identical to spending, but is probably a reasonable proxy.</p>
<p>For 2009, Reuters reports a substantially lower revenue figure than Statista, suggesting Statista figures may be systematically high, e.g. by being relatively inclusive:</p>
<p>“The global computer hardware market had total revenue of $193.2 billion in 2009, representing a compound annual growth rate (CAGR) of 5.4% for the period spanning 2005-2009.” – <a href="http://www.reuters.com/article/idUS230848+13-Jul-2011+BW20110713">Research and Markets press release, Reuters</a>,</p>
<p><a href="http://www.statista.com/statistics/268633/worldwide-revenue-from-computer-hardware-since-2005/">Statista</a>‘s figure indicates revenue of 296 billion Euros, or around $320 billion USD in 2009 (this is the same figure as for 2007, which may be the only number you can see without a subscription—so while it may look like we made an error here, we do have the figure for the correct year). This is around 50% more than the Research and Markets press release.</p>
<p>From these figures we estimate that spending on hardware in 2015 was $300bn-$1,500bn<a class="easy-footnote-to-top" href="#easy-footnote-3-644"></a></p>
</div></li>
<li><div class="li">
<span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-4-644"></span>“It is written from the perspective of the results of our recent inventory of 60 technological categories between 1986 and 2007 (measured in bits and MIPS [million-instructions-per-second])…”
                  <div class="page" title="Page 1">
<div class="layoutArea">
<div class="column">
<p>“…The combined capacity of both groups of computers has grown from 730 tera-IPS in 1986 (73010^12 instructions per second), over 22.5 peta-IPS in 1993 (22.510^15 instructions per second), to 1.8 exa-IPS in 2000 (1.810^18 IPS), and 196 exa-IPS in 2007 (or roughly 210^20 instructions per second).”</p>
</div>
</div>
</div>
<p>–  <a href="http://ijoc.org/index.php/ijoc/article/view/1562/742">Hilbert &amp; Lopez</a> (2012)</p><a class="easy-footnote-to-top" href="#easy-footnote-4-644"></a>
</div></li>
<li><div class="li">
<span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-5-644"></span>In 2007, GWP <a href="https://web.archive.org/web/20080401214624/https://www.cia.gov/library/publications/the-world-factbook/fields/2001.html">was probably</a> about $66T (in 2007 dollars). According to <a href="http://ijoc.org/index.php/ijoc/article/view/1562/742">Hilbert &amp; Lopez</a>, the world could then perform 2 x 10<sup>20</sup> IPS, which is  2 x 10<sup>14</sup> MIPS. According to <a href="https://intelligence.org/2014/05/12/exponential-and-non-exponential/#footnote_7_11027">Muehlhauser &amp; Rieber</a>, hardware cost roughly $5 x 10<sup>-3</sup>/MIPS in 2007. Thus the total value of hardware would have been around $5 x 10<sup>-3</sup>/MIPS x 2 x 10<sup>14</sup> MIPS = $10<sup>12</sup> (a trillion dollars), or 1.5% of GWP.<a class="easy-footnote-to-top" href="#easy-footnote-5-644"></a>
</div></li>
<li><div class="li">
<span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-6-644"></span>“The respective compound annual growth rates between 1986–2007 were 61% for general-purpose computations and 86% for application-specific computations, which is 10 and 14 times faster than global GDP during that period, respectively.”
                  <p><a href="http://ijoc.org/index.php/ijoc/article/view/1562/742">Hilbert &amp; Lopez</a> (2012)</p><a class="easy-footnote-to-top" href="#easy-footnote-6-644"></a>
</div></li>
<li><div class="li">
<span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-7-644"></span>2 x 10<sup>20</sup> /10<sup>18</sup>  = 2 x 10<sup>2</sup>
<p>1.5 x 10<sup>21</sup>/10<sup>18</sup>=1.5 x 10<sup>3</sup><a class="easy-footnote-to-top" href="#easy-footnote-7-644"></a></p>
</div></li>
<li><div class="li">
<span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-8-644"></span>1.5 x 10<sup>21</sup> FLOPS of hardware divided by 3 x 10<sup>13</sup> FLOPS/brain gives us 5 x 10<sup>7</sup> minds.<a class="easy-footnote-to-top" href="#easy-footnote-8-644"></a>
</div></li>
</ol>
</HTML>
AI Impacts Wiki

User Tools

Site Tools

Page Tools
