AI Impacts Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Argument for AI x-risk from large impacts ======

// Published 28 September, 2021; last updated 19 November, 2021 //



<HTML>
<p><em>This page is incomplete, under active work and may be updated soon.</em></p>
</HTML>


<HTML>
<p>The <strong>argument for AI x-risk from large impacts</strong> an argument supporting the claim that advanced artificial intelligence poses an existential risk to humanity.</p>
</HTML>



===== Details =====


==== Argument ====


<HTML>
<p>We understand the argument as follows:</p>
</HTML>


<HTML>
<ol>
<li><div class="li">The creation of advanced AI is likely to have a large impact on the world at the time, relative to other developments.</div></li>
<li><div class="li">The creation of advanced AI is more likely to impact humanity’s long term trajectory than other developments (from 1)</div></li>
<li><div class="li">Developments with large impacts on the future are more likely to be worth influencing, all things equal, than developments with smaller impacts</div></li>
<li><div class="li">One main way things might not be equal is that some developments may be easier to influence than others, but AI currently looks tractable to influence.</div></li>
<li><div class="li">The creation of advanced AI is especially likely to be worth trying to influence, among other developments</div></li>
</ol>
</HTML>


==== Counterarguments ====


<HTML>
<p>This is a neither exhaustive nor exclusive list of counterarguments and complications that we know of:</p>
</HTML>


<HTML>
<ol>
<li><div class="li">This is a heuristic argument that isn’t strictly true, and it isn’t clear how true it is.</div></li>
<li><div class="li">It isn’t clear what a development is. For instance, is global agriculture surviving for another year a development? Is ‘modern science’ a development? Why think of advanced AI as a development, rather than thinking of a particular algorithmic tweak as a development? If we lump more things together, the developments are bigger, so it isn’t clear what it means to say that one is especially big.</div></li>
<li><div class="li">It isn’t clear what counterfactuals are being used. Is the claim that AI will have a large impact relative to a world without any AI? To a world that is similar to the current world forever? Why is the difference between AI and either of those (highly unlikely) scenarios relevant to the value of influencing the development and deployment of AI?</div></li>
<li><div class="li">The heuristic connection between the scale of an action’s possible impact and the stakes of influencing it seems perhaps more useful for ruling out fundamentally unworthy issues than comparing minor impacts on very large issues. For instance, one can use it to confidently tell that it isn’t worth spending half an hour deciding which flavor of jam to buy, whereas it seems much worse for comparing ‘working on functional institutions’ to ‘working on AI’, where both are vaguely ‘big’ and your impact is likely to be ‘tiny’.</div></li>
<li><div class="li">How easy different ‘developments’ are to influence is as big an issue as how much is at stake in influencing them, so is better treated quantitatively, not as a possible qualitative defeater.</div></li>
<li><div class="li">It isn’t clear that ‘influencing developments’ is a good way of viewing possibly high-value actions. For instance, it excludes many actions that are not in response to the actions of others.</div></li>
</ol>
</HTML>


==== Discussion elsewhere ====


<HTML>
<p>We have seen discussion of this in the following places. The name is from Ngo.</p>
</HTML>


<HTML>
<ul>
<li><div class="li"><em>Ngo, Richard. “Thinking Complete: Disentangling Arguments for the Importance of AI Safety.” Thinking Complete (blog), January 21, 2019. <a href="http://thinkingcomplete.blogspot.com/2019/01/disentangling-arguments-for-importance.html">http://thinkingcomplete.blogspot.com/2019/01/disentangling-arguments-for-importance.html</a>.</em></div></li>
<li><div class="li"><em>80,000 Hours. “How Sure Are We about This AI Stuff?” Accessed September 16, 2020. <a href="https://80000hours.org/podcast/episodes/ben-garfinkel-classic-ai-risk-arguments/">https://80000hours.org/podcast/episodes/ben-garfinkel-classic-ai-risk-arguments/</a>.</em></div></li>
</ul>
</HTML>


<HTML>
<p>Richard Ngo describes this as follows<span class="easy-footnote-margin-adjust" id="easy-footnote-1-2661"></span><span class="easy-footnote"><a href="#easy-footnote-bottom-1-2661" title='Ngo, Richard. “Thinking Complete: Disentangling Arguments for the Importance of AI Safety.” &lt;em&gt;Thinking Complete&lt;/em&gt; (blog), January 21, 2019. &lt;a href="http://thinkingcomplete.blogspot.com/2019/01/disentangling-arguments-for-importance.html"&gt;http://thinkingcomplete.blogspot.com/2019/01/disentangling-arguments-for-importance.html&lt;/a&gt;. &lt;br&gt;'><sup>1</sup></a></span>:</p>
</HTML>


<HTML>
<ol>
<li><div class="li">
<em>Argument from large impacts</em>. Even if we’re very uncertain about what AGI development and deployment will look like, it seems likely that AGI will have a very large impact on the world in general, and that further investigation into how to direct that impact could prove very valuable.
                  <ol>
<li><div class="li">Weak version: development of AGI will be at least as big an economic jump as the industrial revolution, and therefore affect the trajectory of the long-term future. See <a href="https://forum.effectivealtruism.org/posts/9sBAW3qKppnoG3QPq/ben-garfinkel-how-sure-are-we-about-this-ai-stuff">Ben Garfinkel’s talk at EA Global London 2018</a>. Ben noted that to consider work on AI safety important, we also need to believe the additional claim that there are feasible ways to positively influence the long-term effects of AI development – something which may not have been true for the industrial revolution. (Personally my guess is that since AI development will happen more quickly than the industrial revolution, power will be more concentrated during the transition period, and so influencing its long-term effects will be more tractable.)
                    </div></li>
<li><div class="li">Strong version: development of AGI will make humans the second most intelligent species on the planet. Given that it was our intelligence which allowed us to control the world to the large extent that we do, we should expect that entities which are much more intelligent than us will end up controlling our future, unless there are reliable and feasible ways to prevent it. So far we have not discovered any.</div></li>
</ol>
</div></li>
</ol>
</HTML>


<HTML>
<p>(We treat 2 as a separate argument, ‘<em><a href="https://wiki.aiimpacts.org/doku.php?id=arguments_for_ai_risk:is_ai_an_existential_threat_to_humanity:will_malign_ai_agents_control_the_future:argument_for_ai_x-risk_from_most_intelligent_species">Argument from most intelligent species</a></em>‘.)</p>
</HTML>


===== Notes =====


<HTML>
<ol class="easy-footnotes-wrapper">
<li><div class="li">
<span class="easy-footnote-margin-adjust" id="easy-footnote-bottom-1-2661"></span>Ngo, Richard. “Thinking Complete: Disentangling Arguments for the Importance of AI Safety.” <em>Thinking Complete</em> (blog), January 21, 2019. <a href="http://thinkingcomplete.blogspot.com/2019/01/disentangling-arguments-for-importance.html">http://thinkingcomplete.blogspot.com/2019/01/disentangling-arguments-for-importance.html</a>.<br/>
<a class="easy-footnote-to-top" href="#easy-footnote-1-2661"></a>
</div></li>
</ol>
</HTML>
AI Impacts Wiki

User Tools

Site Tools

Page Tools
