This page is incomplete, under active work and may be updated soon.

The argument for AI x-risk from powerful black boxes says that very powerful technology that nobody understands is an unprecedented source of existential risk to humanity.

Summary:

1. So far, humans have developed technology largely through understanding relevant mechanisms
2. AI systems developed in 2024 are created via repeatedly modifying random systems in the direction of desired behaviors, rather than manually built, so the mechanisms the systems themselves ultimately use are not understood by human developers
3. Systems whose mechanisms are not understood are more likely to produce undesired consequences than well-understood systems.
4. If such systems are powerful, then the scale of undesired consequences may be catastrophic.

• It is not clear that developing technology without understanding mechanisms is so rare. Some areas such as drug development appear to involve trial and error with little mechanistic understanding.
• This is an argument that risks from the technology are unusually high, but does not say anything about the scale of the effect so does not imply that the risk of risks to humanity as a whole are non-negligible.

Primary author: Katja Grace

Other authors: Nathan Young, Josh Hart

Suggested citation:

Grace, K., Young, N., Hart, J., (2024), Argument for AI x-risk from powerful black boxes, AI Impacts Wiki, https://wiki.aiimpacts.org/arguments_for_ai_risk/argument_for_ai_x-risk_from_powerful_black_boxes