AI Impacts Wiki

User Tools

Site Tools

You are here: Welcome to the AI Impacts Wiki! » uncategorized » AI Safety Arguments Affected by Chaos
uncategorized:ai_safety_arguments_affected_by_chaos

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
uncategorized:ai_safety_arguments_affected_by_chaos [2023/04/05 19:54]
jeffreyheninger 		uncategorized:ai_safety_arguments_affected_by_chaos [2023/04/12 22:32] (current)
katjagrace [AI Safety Arguments Affected by Chaos]
Line 2: Line 2:
  
 //Created 31 March, 2023. Last updated 31 March, 2023.// //Created 31 March, 2023. Last updated 31 March, 2023.//
 +
 +//This page is under review and may be updated soon.//
  
 Chaos theory allows us to show that some predictions cannot be reliably made, even using arbitrary intelligence. Some things about human brains seem to be in that category, which affects how advanced AI might interact with humans. Chaos theory allows us to show that some predictions cannot be reliably made, even using arbitrary intelligence. Some things about human brains seem to be in that category, which affects how advanced AI might interact with humans.
Line 22: Line 24:
 A major reason why we might care about how much headroom there is above human intelligence is to attempt to understand takeover scenarios. If a superintelligent AI were to try to wrest control of the future away from humanity, how much chance would we have of preventing it? If humans were close to the ceiling on lots of skills at this point, perhaps aided by narrow AI, then we might not have that much of a disadvantage. If humans were far from the ceiling on many important skills, then we would expect to be at a serious disadvantage.((Eliezer Yudkowsky. //My Childhood Role Model.// LessWrong. (2008) [[https://www.lesswrong.com/posts/3Jpchgy53D2gB5qdk/my-childhood-role-model]].)) An argument to this effect has been made by Barak and Edelman, and is discussed below. A major reason why we might care about how much headroom there is above human intelligence is to attempt to understand takeover scenarios. If a superintelligent AI were to try to wrest control of the future away from humanity, how much chance would we have of preventing it? If humans were close to the ceiling on lots of skills at this point, perhaps aided by narrow AI, then we might not have that much of a disadvantage. If humans were far from the ceiling on many important skills, then we would expect to be at a serious disadvantage.((Eliezer Yudkowsky. //My Childhood Role Model.// LessWrong. (2008) [[https://www.lesswrong.com/posts/3Jpchgy53D2gB5qdk/my-childhood-role-model]].)) An argument to this effect has been made by Barak and Edelman, and is discussed below.
  
-For many tasks, having a high skill level requires you to be able to make predictions about what will happen in the future, contingent on what choices you choose to make. Chaos theory provides a way to prove that making reliable predictions about certain things is impossible for an arbitrary intelligence given a small amount of initial uncertainty. For these predictions, the skill ceiling is low. It is often still possible to make a prediction about something, perhaps the statistics of the motion or perhaps something else even less related to the original question, but not to predict what exactly will happen in the future.((What sorts of predictions can and cannot be made when there is chaos is discussed in Section 7 of the accompanying report. \\ Heninger & Johnson. //Chaos and Intrinsic Unpredictability.// AI Impacts. [[http://aiimpacts.org/wp-content/uploads/2023/03/Chaos-and-Intrinsic-Unpredictability.pdf]].)) Whenever predicting chaotic motion is important, we should not expect that AI will be able to perform arbitrarily well.+For many tasks, having a high skill level requires you to be able to make predictions about what will happen in the future, contingent on what choices you choose to make. Chaos theory provides a way to prove that making reliable predictions about certain things is impossible for an arbitrary intelligence given a small amount of initial uncertainty. For these predictions, the skill ceiling is low. It is often still possible to make a prediction about something, perhaps the statistics of the motion or perhaps something else even less related to the original question, but not to predict what exactly will happen in the future.((What sorts of predictions can and cannot be made when there is chaos is discussed in Section 7 of the accompanying report. \\ Heninger & Johnson. //Chaos and Intrinsic Unpredictability.// AI Impacts. [[http://aiimpacts.org/wp-content/uploads/2023/04/Chaos-and-Intrinsic-Unpredictability.pdf]].)) Whenever predicting chaotic motion is important, we should not expect that AI will be able to perform arbitrarily well.
  
 Along with proving the existence of some skill ceilings, chaos theory might also help us better understand the skill landscape: How much intelligence is required to achieve a particular skill level? Knowing what the steepness of this slope is (the marginal improvement of skill with additional intelligence) would inform expectations of takeoff speed, and what we might expect from interactions with AIs which are slightly more intelligent than humans. Along with proving the existence of some skill ceilings, chaos theory might also help us better understand the skill landscape: How much intelligence is required to achieve a particular skill level? Knowing what the steepness of this slope is (the marginal improvement of skill with additional intelligence) would inform expectations of takeoff speed, and what we might expect from interactions with AIs which are slightly more intelligent than humans.
Line 44: Line 46:
 There are many things that humans have difficulty predicting. For some of these things, better predictions are possible, if only we were more intelligent. For others, there are physical reasons why we are incapable of predicting them. If there is something that we cannot predict because of chaos, an arbitrarily intelligent AI would not be able to predict it either. There are many things that humans have difficulty predicting. For some of these things, better predictions are possible, if only we were more intelligent. For others, there are physical reasons why we are incapable of predicting them. If there is something that we cannot predict because of chaos, an arbitrarily intelligent AI would not be able to predict it either.
  
-The classic example of chaos in nature is the weather. Predicting the weather more than 10 days out is impossible. It is possible to make some statistical predictions, most often by looking at averages of historical weather data. Despite being chaotic, weather is still partially controllable, by seeding clouds for example.((What it means for a chaotic system to be controllable is discussed in Section 8 of the accompanying report. \\ Heninger & Johnson. //Chaos and Intrinsic Unpredictability.// AI Impacts. [[http://aiimpacts.org/wp-content/uploads/2023/03/Chaos-and-Intrinsic-Unpredictability.pdf]].)) In order to control the weather, you have to adjust what inputs you are using daily, in order to continually respond to the growing uncertainties.+The classic example of chaos in nature is the weather. Predicting the weather more than 10 days out is impossible. It is possible to make some statistical predictions, most often by looking at averages of historical weather data. Despite being chaotic, weather is still partially controllable, by seeding clouds for example.((What it means for a chaotic system to be controllable is discussed in Section 8 of the accompanying report. \\ Heninger & Johnson. //Chaos and Intrinsic Unpredictability.// AI Impacts. [[http://aiimpacts.org/wp-content/uploads/2023/04/Chaos-and-Intrinsic-Unpredictability.pdf]].)) In order to control the weather, you have to adjust what inputs you are using daily, in order to continually respond to the growing uncertainties.
  
 Many natural disasters are weather events, including hurricanes and droughts, so they are similarly hard to predict. Some other natural disasters are caused by chaotic systems too. Solar storms are caused by turbulence in the sun’s atmosphere. The convection in the mantle driving earthquakes and volcanoes might also be chaotic, although the Lyapunov time seems unlikely to be less than 100,000 years,((One unusually fast geological time scale is geomagnetic reversal, which happens about every 500,000 years. This is caused by turbulence in the outer core, which is less viscous than the mantle. The Lyapunov time for convection in the mantle seems likely to be tens of millions of years.)) so chaos theory does not restrict our predictions here on human-relative time scales. Volcanic eruptions typically do have precursors, and so can be predicted. Earthquakes are harder to predict, both because it is hard to measure what is happening inside a fault and because the slow dynamics of the mantle interact with a much faster time scale: how long it takes for rocks to break. Many natural disasters are weather events, including hurricanes and droughts, so they are similarly hard to predict. Some other natural disasters are caused by chaotic systems too. Solar storms are caused by turbulence in the sun’s atmosphere. The convection in the mantle driving earthquakes and volcanoes might also be chaotic, although the Lyapunov time seems unlikely to be less than 100,000 years,((One unusually fast geological time scale is geomagnetic reversal, which happens about every 500,000 years. This is caused by turbulence in the outer core, which is less viscous than the mantle. The Lyapunov time for convection in the mantle seems likely to be tens of millions of years.)) so chaos theory does not restrict our predictions here on human-relative time scales. Volcanic eruptions typically do have precursors, and so can be predicted. Earthquakes are harder to predict, both because it is hard to measure what is happening inside a fault and because the slow dynamics of the mantle interact with a much faster time scale: how long it takes for rocks to break.
Line 55: Line 57:
 Simple food chains can exhibit some interesting dynamics in the size of populations of various species.((The classic example involves a 220 year dataset of the number of lynx and snowshoe hare pelts caught by the Hudson Bay Company. \\ May. //Stability and Complexity in Model Ecosystems.// (Princeton University Press, 1974) )) More complicated food webs involving many species likely can be chaotic, although it is hard to distinguish this from changes in the population as a result of a chaos in the environment. Simple food chains can exhibit some interesting dynamics in the size of populations of various species.((The classic example involves a 220 year dataset of the number of lynx and snowshoe hare pelts caught by the Hudson Bay Company. \\ May. //Stability and Complexity in Model Ecosystems.// (Princeton University Press, 1974) )) More complicated food webs involving many species likely can be chaotic, although it is hard to distinguish this from changes in the population as a result of a chaos in the environment.
  
-Markets also involve many actors with complicated interactions, so it seems likely that there is chaos involved to some extent. Since people have incentives to look for arbitrary patterns and to respond to them, it is probably better to model markets as anti-inductive.((The difference between chaotic and anti-inductive systems is explained in Section 2 of the accompanying report. \\ Heninger & Johnson. //Chaos and Intrinsic Unpredictability.// AI Impacts. [[http://aiimpacts.org/wp-content/uploads/2023/03/Chaos-and-Intrinsic-Unpredictability.pdf]].))+Markets also involve many actors with complicated interactions, so it seems likely that there is chaos involved to some extent. Since people have incentives to look for arbitrary patterns and to respond to them, it is probably better to model markets as anti-inductive.((The difference between chaotic and anti-inductive systems is explained in Section 3 of the accompanying report. \\ Heninger & Johnson. //Chaos and Intrinsic Unpredictability.// AI Impacts. [[http://aiimpacts.org/wp-content/uploads/2023/04/Chaos-and-Intrinsic-Unpredictability.pdf]].))
  
 Perhaps the most interesting potentially chaotic thing is the human brain. It is discussed in the next section.  Perhaps the most interesting potentially chaotic thing is the human brain. It is discussed in the next section. 
Line 65: Line 67:
 For the things for which a brain is chaotic, it is impossible to predict what in particular that brain will do. A simulation of all of the activations of all of the neurons in the brain, or a copy of the brain made as accurately as is physically possible, will not continue to accurately model the behavior of that brain for more than a second into the future. For the things for which a brain is chaotic, it is impossible to predict what in particular that brain will do. A simulation of all of the activations of all of the neurons in the brain, or a copy of the brain made as accurately as is physically possible, will not continue to accurately model the behavior of that brain for more than a second into the future.
  
-Even when predicting what in particular a brain will do is impossible, it might still be possible to make statistical predictions. Knowing the statistics would allow you to construct a probability distribution over possible future behaviors of the brain. Human-like behavior could be sampled from the distribution. It is not obvious if this could actually be done, both because the distribution could be spread over an extremely large space and because the distribution itself could also change chaotically and so be unpredictable.((This is explained in more detail in Section 7 of the accompanying report. \\ Heninger & Johnson. //Chaos and Intrinsic Unpredictability.// AI Impacts. [[http://aiimpacts.org/wp-content/uploads/2023/03/Chaos-and-Intrinsic-Unpredictability.pdf]].)) Figuring out whether the motion of the distribution is chaotic is much harder, so this page will not make strong claims about it.+Even when predicting what in particular a brain will do is impossible, it might still be possible to make statistical predictions. Knowing the statistics would allow you to construct a probability distribution over possible future behaviors of the brain. Human-like behavior could be sampled from the distribution. It is not obvious if this could actually be done, both because the distribution could be spread over an extremely large space and because the distribution itself could also change chaotically and so be unpredictable.((This is explained in more detail in Section 7 of the accompanying report. \\ Heninger & Johnson. //Chaos and Intrinsic Unpredictability.// AI Impacts. [[http://aiimpacts.org/wp-content/uploads/2023/04/Chaos-and-Intrinsic-Unpredictability.pdf]].)) Figuring out whether the motion of the distribution is chaotic is much harder, so this page will not make strong claims about it.
  
 This argument might feel like it proves too much. Some aspects of human behavior are clearly predictable some of the time. There are several ways this argument should be tempered to make it consistent with this common experience: (1) For some things, the relevant parts of the human brain are not being chaotic. (2) Some of the chaos in the brain might have predictable statistics. Behavior which depends on statistics which are stationary and not multistable can be predictable. (3) Your own brain has a similar causal structure / coarse-graining as the person you are trying to predict. Using empathetic inference to model their behavior is more likely to result in something similar to their behavior than a model built with a very different causal structure / coarse-grainings. Even with these caveats, it still seems likely that there are some aspects of human behavior which are inherently unpredictable. This argument might feel like it proves too much. Some aspects of human behavior are clearly predictable some of the time. There are several ways this argument should be tempered to make it consistent with this common experience: (1) For some things, the relevant parts of the human brain are not being chaotic. (2) Some of the chaos in the brain might have predictable statistics. Behavior which depends on statistics which are stationary and not multistable can be predictable. (3) Your own brain has a similar causal structure / coarse-graining as the person you are trying to predict. Using empathetic inference to model their behavior is more likely to result in something similar to their behavior than a model built with a very different causal structure / coarse-grainings. Even with these caveats, it still seems likely that there are some aspects of human behavior which are inherently unpredictable.
uncategorized/ai_safety_arguments_affected_by_chaos.1680724480.txt.gz · Last modified: 2023/04/05 19:54 by jeffreyheninger

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC0 1.0 Universal
CC0 1.0 Universal Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki