AI Impacts Wiki arguments_for_ai_risk

Incentives to create AI systems known to pose extinction risks

2023-06-08T21:46:20+00:00

@@ -49,9 +49,9 @@
  
  
  <HTML>
  <ol>
- <li><div class="li">A person faces the choice of using an AI lawyer system for $100, or a human lawyer for $10,000. They believe that the AI lawyer system is poorly motivated and agentic, and that movement of resources to such systems is gradually disempowering humanity, which they care about. Nonetheless, their action only contributes a small amount to this problem, and they are not willing to raise tens of thousands of dollars to avoid that harm.</div></li>
+ <li><div class="li">A person faces the choice of using an AI lawyer system for \$100, or a human lawyer for \$10,000. They believe that the AI lawyer system is poorly motivated and agentic, and that movement of resources to such systems is gradually disempowering humanity, which they care about. Nonetheless, their action only contributes a small amount to this problem, and they are not willing to raise tens of thousands of dollars to avoid that harm.</div></li>
  <li><div class="li">A person faces the choice of deploying the largest scale model to date, or trying to call off the project. They believe that at some scale, a model will become an existential threat to humanity. However they are very unsure at what scale, and estimate that the model in front of them only has a 1% chance of being the dangerous one. They value the future of humanity a lot, but not ten times more than their career, and calling off the project would be a huge hit, for only 1% of the future of humanity.</div></li>
  </ol>
  </HTML>

Interviews on plausibility of AI safety by default

2022-09-21T07:37:41+00:00

@@ -1 +1,42 @@
+ ====== Interviews on plausibility of AI safety by default ======
+ 
+ // Published 02 April, 2020; last updated 15 September, 2020 //
+ 
+ <HTML>
+ <p>This is a list of interviews on the plausibility of AI safety by default.</p>
+ </HTML>
+ 
+ 
+ ===== Background =====
+ 
+ 
+ <HTML>
+ <p>AI Impacts conducted interviews with several thinkers on AI safety in 2019 as part of a project exploring arguments for expecting advanced AI to be safe by default. The interviews also covered other AI safety topics, such as timelines to advanced AI, the likelihood of current techniques leading to AGI, and currently promising AI safety interventions.</p>
+ </HTML>
+ 
+ 
+ ===== List =====
+ 
+ 
+ <HTML>
+ <ul>
+ <li><div class="li">
+ <a href="/doku.php?id=conversation_notes:conversation_with_ernie_davis">Conversation with Ernie Davis</a>
+ </div></li>
+ <li><div class="li">
+ <a href="/doku.php?id=conversation_notes:conversation_with_rohin_shah">Conversation with Rohin Shah</a>
+ </div></li>
+ <li><div class="li">
+ <a href="/doku.php?id=conversation_notes:conversation_with_paul_christiano">Conversation with Paul Christiano</a>
+ </div></li>
+ <li><div class="li">
+ <a href="/doku.php?id=conversation_notes:conversation_with_adam_gleave">Conversation with Adam Gleave</a>
+ </div></li>
+ <li><div class="li">
+ <a href="/doku.php?id=conversation_notes:conversation_with_robin_hanson">Conversation with Robin Hanson</a>
+ </div></li>
+ </ul>
+ </HTML>
+ 
+

List of possible risks from AI

2023-09-26T17:00:58+00:00

@@ -1,5 +1,5 @@
- ====== List of Possible Risks from AI ======
+ ====== List of possible risks from AI ======
  
  //Created 26 September, 2023. Last updated 26 September, 2023.//
  
  Many people are concerned about AI for reasons other than existential risk. Here we list some of these other concerns.((Many of these concerns were raised in response to a tweet by Katja Grace: [[https://twitter.com/KatjaGrace/status/1693873324747862370|https://twitter.com/KatjaGrace/status/1693873324747862370]].))

List of sources arguing against existential risk from AI

2023-06-16T22:39:54+00:00

@@ -14,10 +14,9 @@
  </HTML>
  
  
  
- ===== 
- List =====
+ ===== List =====
  
  
  <HTML>
  <p><strong>Cegłowski, Maciej. “Superintelligence: The Idea That Eats Smart People.”</strong> <em>Idle Words</em> (blog). Accessed December 9, 2021. <a href="https://idlewords.com/talks/superintelligence.htm">https://idlewords.com/talks/superintelligence.htm</a>.</p>

List of sources arguing for existential risk from AI

2023-06-08T21:39:20+00:00

@@ -118,25 +118,16 @@
  
  ===== See also =====
  
  
- <HTML>
- <ul>
- <li><div class="li">
- <a href="/doku.php?id=arguments_for_ai_risk:list_of_sources_arguing_against_existential_risk_from_ai">List of sources arguing against existential risk from AI</a>
- </div></li>
- <li><div class="li">
- <a href="https://aiimpacts.org/does-ai-pose-an-existential-risk/">Is AI an existential threat to humanity?</a>
- </div></li>
- </ul>
- </HTML>
+   * [[arguments_for_ai_risk:list_of_sources_arguing_against_existential_risk_from_ai|List of sources arguing against existential risk from AI]]
+   * [[arguments_for_ai_risk:is_ai_an_existential_threat_to_humanity:start|Is AI an existential risk to humanity?]]
  
  
- <HTML>
- <p><em>Primary author: Katja Grace</em></p>
- </HTML>
+ //Primary author: Katja Grace//
+ 
  
  
  ===== Notes =====

Quantitative Estimates of AI Risk

2023-12-01T18:15:56+00:00

@@ -2,9 +2,9 @@
  /*
  COMMENT:
  Things to add to this:
  - https://optimists.ai/2023/11/28/ai-is-easy-to-control/
- - Katja's EAG 2023 talk, where she says 19% chance
+ 
  */
  // This page is in an early draft. It is very incomplete and may contain errors. // 
  
  Some people who are working in AI Safety have published quantitative estimates for how likely they think it is that AI will pose an existential threat.

Existential risk from AI

2024-07-16T18:05:27+00:00

@@ -1,5 +1,5 @@
- ====== Existential risk from AI portal ======
+ ====== Existential risk from AI ======
  
  Pages on this topic include:
    * [[arguments_for_ai_risk:is_ai_an_existential_threat_to_humanity:start|Is AI an existential threat to humanity?]]
    * [[will_superhuman_ai_be_created:start|Will superhuman AI be created?]]

Stuart Russell’s description of AI risk

2022-09-21T07:37:41+00:00

@@ -1 +1,38 @@
+ ====== Stuart Russell’s description of AI risk ======
+ 
+ // Published 11 September, 2017; last updated 28 May, 2020 //
+ 
+ <HTML>
+ <p>Stuart Russell has argued that advanced AI poses a risk, because it will have the ability to make high quality decisions, yet may not share human values perfectly.</p>
+ </HTML>
+ 
+ 
+ ===== Details =====
+ 
+ 
+ <HTML>
+ <p>Stuart Russell describes a risk from highly advanced AI <a href="https://www.edge.org/conversation/the-myth-of-ai#26015">here</a>. In short:</p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p style="padding-left: 30px;">The primary concern [with highly advanced AI] is not spooky emergent consciousness but simply the ability to make high-quality decisions. Here, quality refers to the expected outcome utility of actions taken […]. Now we have a problem:</p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p style="padding-left: 60px;">1. The utility function may not be perfectly aligned with the values of the human race, which are (at best) very difficult to pin down.</p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p style="padding-left: 60px;">2. Any sufficiently capable intelligent system will prefer to ensure its own continued existence and to acquire physical and computational resources – not for their own sake, but to succeed in its assigned task.</p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p style="padding-left: 30px;">A system that is optimizing a function of n variables, where the objective depends on a subset of size k&lt;n, will often set the remaining unconstrained variables to extreme values; if one of those unconstrained variables is actually something we care about, the solution found may be highly undesirable.  This is essentially the old story of the genie in the lamp, or the sorcerer’s apprentice, or King Midas: you get exactly what you ask for, not what you want.</p>
+ </HTML>
+ 
+

Views of prominent AI developers on risk from AI

2024-07-22T16:24:29+00:00

@@ -1,7 +1,7 @@
  ====== Views of prominent AI developers on risk from AI ======
  
- //This page is in an early draft. It is incomplete and may contain errors.//
+ //This page is a draft, and out of date. It is incomplete and may contain errors.//
  
  People who have worked on creating artificial intelligence have a variety of views on risk from AI, both for the potential benefits and potential downsides.
  
  ===== Background =====