| Both sides previous revision
Previous revision
Next revision
|
Previous revision
|
arguments_for_ai_risk:is_ai_an_existential_threat_to_humanity:start [2023/02/12 04:37]
katjagrace [Scenario: amoral AI agents control the future] |
arguments_for_ai_risk:is_ai_an_existential_threat_to_humanity:start [2023/09/26 17:01] (current)
jeffreyheninger |
| ====== Is AI an existential threat to humanity? ====== | ====== Is AI an existential risk to humanity? ====== |
| |
| //This page is under active work and may be updated soon.// | //This page is under active work and may be updated soon.// |
| |
| The balance of evidence appears to suggest that AI poses a substantial existential risk, though none of the arguments that we know of appear be strongly compelling evidence. | The balance of evidence appears to suggest that AI poses a substantial existential risk, though none of the arguments that we know of appear be conclusive evidence. |
| |
| ===== Background ===== | ===== Background ===== |
| |
| Many [[arguments_for_ai_risk:list_of_sources_arguing_for_existential_risk_from_ai|thinkers]] believe advanced [[clarifying_concepts:artificial_intelligence|artificial intelligence]] (AI) poses a large threat to humanity's long term survival or flourishing. Here we review evidence. | Many [[arguments_for_ai_risk:list_of_sources_arguing_for_existential_risk_from_ai|thinkers]] believe advanced [[clarifying_concepts:artificial_intelligence|artificial intelligence]] (AI) poses a large threat to humanity's long term survival or flourishing. Here we review evidence. |
| | |
| | For views of specific people working on AI, see [[arguments_for_ai_risk:views_of_ai_developers_on_risk_from_ai|this page]]. |
| |
| Note that arguments included here are not intended to be straightforwardly independent lines of evidence. They may instead represent different ways of conceptualizing and reasoning about the same underlying situation. | Note that arguments included here are not intended to be straightforwardly independent lines of evidence. They may instead represent different ways of conceptualizing and reasoning about the same underlying situation. |
| Advanced AI could conceivably threaten humanity's future via several different disaster scenarios, each of which is suggested by different arguments. We consider these scenarios and arguments here, one at a time. | Advanced AI could conceivably threaten humanity's future via several different disaster scenarios, each of which is suggested by different arguments. We consider these scenarios and arguments here, one at a time. |
| |
| ==== Scenario: amoral AI agents control the future ==== | ==== Scenario: malign AI agents control the future ==== |
| |
| //(Main article: [[arguments_for_ai_risk:is_ai_an_existential_threat_to_humanity: will_malign_ai_agents_control_the_future:start|Will malign AI agents control the future?]]) | //(Main article: [[arguments_for_ai_risk:is_ai_an_existential_threat_to_humanity: will_malign_ai_agents_control_the_future:start|Will malign AI agents control the future?]]) |
| This scenario includes sub-scenarios where the above process happens fast or slow, or involves different kinds of agents, or different specific routes, etc. | This scenario includes sub-scenarios where the above process happens fast or slow, or involves different kinds of agents, or different specific routes, etc. |
| |
| Arguments for this scenario occurring include: | Various [[https://wiki.aiimpacts.org/doku.php?id=arguments_for_ai_risk:is_ai_an_existential_threat_to_humanity:will_malign_ai_agents_control_the_future:start#arguments|arguments]] are made for this scenario. The most prominent appears to be: |
| | |
| * **AI developments will produce powerful agents with undesirable goals** \\ \\ //(Main article: [[arguments_for_ai_risk:is_ai_an_existential_threat_to_humanity:will_malign_ai_agents_control_the_future:argument_for_ai_x-risk_from_competent_malign_agents|Argument for AI X-risk from competent malign agents]])// \\ \\ **Summary**: At least some advanced AI systems will probably be 'goal-oriented', a powerful force in the world, and their goals will probably be bad by human lights. Powerful goal-oriented agents tend to achieve their goals. \\ \\ **Apparent status**: This seems to us the most suggestive argument, though not watertight. This seems prima facie plausible, but destroying everything is a very implausible event, so the burden of proof is high. | |
| | |
| * **AI will replace humans as most intelligent 'species'** \\ \\ //(Main article: [[arguments_for_ai_risk:is_ai_an_existential_threat_to_humanity:will_malign_ai_agents_control_the_future:argument_for_ai_x-risk_from_most_intelligent_species|Argument for AI x-risk from most intelligent species]])// \\ \\ **Summary**: Humans' dominance over other species in controlling the world is due primarily to our superior cognitive abilities. If another 'species' with better cognitive abilities appeared, we should then expect humans to lose control over the future and therefore for the future to lose its value. \\ \\ **Apparent status**: Somewhat suggestive, though doesn't appear to be valid, since intelligence in animals doesn't appear to generally relate to dominance. A valid version may be possible to construct. | |
| | |
| * **AI agents will cause humans to 'lose control'** \\ \\ **Summary**: AI will ultimately be much faster and more competent than humans, so either, a) must make most decisions because waiting for humans will be so costly, b) will make decisions if it wants, since humans will be so relatively powerless, due to their intellectual inferiority. Losing control of the future isn't necessarily bad, but is prima facie a very bad sign. \\ \\ **Apparent status**: Suggestive, but as stated does not appear to be valid. For instance, humans do not generally seem to become disempowered by possession of software that is far superior to them. | |
| | |
| * **Argument for loss of control from extreme speed** \\ \\ **Summary**: Advancing AI will tend to produce very rapid changes, either because of feedback loops in automation of automation processes, or because automation tends to be faster than the human activity it replaces. Faster change reduces human ability to steer a situation, e.g. reviewing and understanding it, responding to problems as they appear, preparing. In the extreme, the pace of socially relevant events could become so fast as to exclude human participation. \\ \\ **Apparent status**: Heuristically suggestive, however the burden of proof should arguably be high for an implausible event such as the destruction of humanity. This argument also seems to support concern about a wide range of technologies, which may be correct. | |
| |
| In light of these arguments, this scenario seems to us plausible but not guaranteed. Its likelihood appears to depend strongly on the strength of one's prior probability on arbitrary risks being sufficient to destroy the world. | * **AI developments will produce powerful agents with undesirable goals** \\ \\ //(Main article: [[arguments_for_ai_risk:is_ai_an_existential_threat_to_humanity:will_malign_ai_agents_control_the_future:argument_for_ai_x-risk_from_competent_malign_agents:start|Argument for AI X-risk from competent malign agents]])// \\ \\ **Summary**: At least some advanced AI systems will probably be 'goal-oriented', a powerful force in the world, and their goals will probably be bad by human lights. Powerful goal-oriented agents tend to achieve their goals. \\ \\ **Apparent status**: This seems to us the most suggestive argument, though not watertight. This seems prima facie plausible, but destroying everything is a very implausible event, so the burden of proof is high. |
| |
| ==== Scenario: bad human actors are empowered by cheap AI cognitive labor ==== | In light of [[https://wiki.aiimpacts.org/doku.php?id=arguments_for_ai_risk:is_ai_an_existential_threat_to_humanity:will_malign_ai_agents_control_the_future:start#arguments|arguments]], this scenario seems plausible but not guaranteed. |
| | ==== Scenario: AI empowers bad human actors ==== |
| |
| Some people and collectives have goals whose fulfillment would be considered bad by most people. If advanced AI empowered those people disproportionately, this could be destructive. This could happen by bad luck, or because the situation systematically advantages unpopular values. | Some people and collectives have goals whose fulfillment would be considered bad by most people. If advanced AI empowered those people disproportionately, this could be destructive. This could happen by bad luck, or because the situation systematically advantages unpopular values. |
| * [[arguments_for_ai_risk:list_of_sources_arguing_for_existential_risk_from_ai|List of sources arguing for existential risk from AI]] | * [[arguments_for_ai_risk:list_of_sources_arguing_for_existential_risk_from_ai|List of sources arguing for existential risk from AI]] |
| * [[will_superhuman_ai_be_created:start|Will superhuman AI be created?]] | * [[will_superhuman_ai_be_created:start|Will superhuman AI be created?]] |
| | * [[arguments_for_ai_risk:list_of_possible_risks_from_ai|List of possible risks from AI]] |
| |
| ====== Notes ====== | ====== Notes ====== |
| |
