AI Impacts Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Examples of Warning Signs ======

//Published: 19 December, 2023; Last Updated: 19 December, 2023.//

//Epistemic Status: Varies. Some of the examples here come from thorough case studies for our other projects. We are less confident of the examples which were not part of a dedicated case study: some of the sources cited were not read thoroughly, and we did not check to make sure that they are representative of the relevant literature.//

This page lists some examples of warning signs from the history of technologies other than AI.

===== Model =====

Sometimes, to understand the development of an emerging technology and the policy surrounding it, it is important to understand a few key events, called 'warning signs.' Activists use these events to argue that the specific technology, or the industry more generally, has important risks that need to be addressed. This galvanizes public support and puts pressure on policy makers. The policies enacted in the wake of these events are important for that industry for at least decades to come.

===== Definitions =====

There are multiple terms that are used to refer to these key events in the history of technology policy. The terms do not mean exactly the same thing, and are used inconsistently. 

"A **warning shot** is a global catastrophe that indirectly reduces existential risk by increasing concern about future catastrophes."((//Warning shot.// EA Forum. (Accessed November 16, 2023.) [[https://forum.effectivealtruism.org/topics/warning-shot]].)) To be a warning shot under this definition, large scale harm has to occur.

A **fire alarm** is something which creates "common knowledge that action is now due and socially acceptable."((Eliezer Yudkowsky. //There's No Fire Alarm for Artificial General Intelligence.// MIRI. (2017) [[https://intelligence.org/2017/10/13/fire-alarm/]].)) The focus here is the creation of common knowledge, rather than the harm caused.

A **warning sign** is "any event that increases concern about a particular category of existential risk, regardless of whether the event itself constitutes a global catastrophe. ... Note, however, that both 'warning shot' and 'fire alarm' are sometimes used as synonyms for 'warning sign.'"((//Warning shot.// EA Forum. (Accessed November 16, 2023.) [[https://forum.effectivealtruism.org/topics/warning-shot]].))

A **trigger event** is "a 'highly publicized, shocking incident' that 'dramatically reveals a critical social problem to the public in a vivid way.'"((//Mass Protest.// Ayni Institute. (Accessed November 16, 2023.) [[https://ayni.institute/massprotest/]].)) This term is not specific to technology policy, and comes from the general theory of protest. A related term is a **moment of the whirlwind**. The focus of this term is on the decision maker at a critical moment, rather than the event leading to that critical moment.

A **Sputnik moment** is "a trigger mechanism, an event that makes people collectively say that they need to do something, and this sets a course in another direction," especially when a country recognizes that its competitors have some technological ability that they do not currently have.((Remy Melina. //What Exactly Is a 'Sputnik Moment?'// Space.com. (2010) [[https://www.space.com/10437-sputnik-moment.html]]].))

A **galvanizing event ** is an event that stimulates people's willingness to take action.

===== Examples =====

Below are some examples of warning signs for various industries: nuclear power, fossil fuels, chlorofluorocarbons, genetically modified organisms, medical research involving viruses, and medical research involving humans. For some of them, the main concern is not existential risk. Some of them also satisfy the definitions of several of the terms described above.

The examples of warning signs are grouped by the industry they were used to promote policies for. Each example also includes the date the warning sign was made known to the public, the location where the warning sign took place, and a brief description of what occurred. The relevance of the warning sign to the problem activists were trying to address is categorized as High, Moderate, or Low. The amount of harm caused is categorized as harm occurring locally, regionally, or globally, and as catastrophic if it kills a majority of the people it impacts or renders a large area uninhabitable, significant if it kills some people or has severe environmental consequences, and low if no one is killed by the warning shot. The resulting policies that this warning sign contributed to are then briefly described.

==== Nuclear Power ====
//[[responses_to_ai:technological_inevitability:incentivized_technologies_not_pursued:nuclear_power|Case Study]]//

=== Three Mile Island ===

  * **Date:** 1979.
  * **Location:** Pennsylvania, USA.
  * **Description:** Partial meltdown at a nuclear power plant caused the release of some radioactive gas and iodine.
  * **Relevance:** High.
  * **Harm Caused:** Too small to measure.
  * **Result:** Reinforced already increasing safety regulations. The Nuclear Regulatory Commission was founded in 1975, and many planned nuclear power plants had already been canceled. No new nuclear power plants would begin construction in the USA for decades.((Hultman and Koomey. //Three Mile Island: The driver of US nuclear power’s decline?// Bulletin of the Atomic Scientists **69.3**. (2013) p. 63-70. [[https://journals.sagepub.com/doi/pdf/10.1177/0096340213485949]].))

=== Chernobyl ===

  * **Date:** 1986.
  * **Location:** Ukraine, USSR.
  * **Description:** Steam explosion, meltdown, and reactor core fire destroyed the containment building and spread radioactive airborne contaminants across Europe. About 30 people died within 3 months, and the WHO estimates about 9,000 excess cancer deaths afterwards, although this is disputed.((//World Health Organization report explains the health impacts of the world's worst-ever civil nuclear accident.// WHO. (2006) [[https://web.archive.org/web/20110404181327/http://www.who.int/mediacentre/news/releases/2006/pr20/en/index.html]].))
  * **Relevance:** High.
  * **Harm Caused:** Catastrophic locally, significant regionally.
  * **Result:** Public opinion in Europe shifts dramatically against nuclear power. Italian referendum results in the closure of all nuclear power plants in Italy.((//1987 Italian referendum.// Wikipedia. (Accessed November 16, 2023.) [[https://en.wikipedia.org/wiki/1987_Italian_referendums]]. \\
//Eligendo Archivio.// Ministero dell’Interno. (Accessed November 16, 2023.) [[https://elezionistorico.interno.gov.it/index.php?tpel=F&dtel=08/11/1987&tpa=I&tpe=A&lev0=0&levsut0=0&es0=S&ms=S]].
)) The USSR slows nuclear power plant construction and commits to increased transparency around nuclear power.((Vyacheslav S. Romanov. //Nuclear power and public opinion.// IAEA Bulletin **2**. (1990) [[https://www.iaea.org/sites/default/files/publications/magazines/bulletin/bull32-2/32205091922.pdf]].)) The IAEA creates two new treaties: the Convention on Early Notification of a Nuclear Accident and the Convention on Assistance in the Case of a Nuclear Accident or Radiological Emergency.

=== Fukushima ===

  * **Date:** 2011.
  * **Location:** Japan.
  * **Description:** Major earthquake and tsunami damaged the electrical grid and the power plant’s backup power, removing the ability to cool the reactors, leading to hydrogen explosions and the release of radioactive contamination into the environment. Linear no-threshold models estimate 130 excess cancer deaths, although this is likely below detectable levels.((Ten Hoeve and Jacobson. //Worldwide health effects of the Fukushima Daiichi nuclear accident.// Energy & Environmental Science. (2012) [[https://web.stanford.edu/group/efmh/jacobson/TenHoeveEES12.pdf]].))
  * **Relevance:** High.
  * **Harm Caused:** Significant locally.
  * **Result:** Japan shuts down all of its nuclear reactors, and later reopens some.((//In a reversal, Japan embraces nuclear power after promising to phase it out.// NPR. (2022) [[https://www.npr.org/2022/12/22/1144990722/japan-nuclear-power-change-fukushima]].)) Germany decides to shut down all of its nuclear reactors.((Eddy and Solomon. //Germany Quits Nuclear Power, Ending a Decades-Long Struggle.// New York Times. (2023) [[https://www.nytimes.com/2023/04/14/world/europe/germany-nuclear-power-plants.html]].))

==== Fossil Fuels ====

=== Exxon Valdez Oil Spill ===

  * **Date:** 1989.
  * **Location:** Alaska, USA.
  * **Description:** An oil tanker struck a reef and spilled 10 million gallons of crude oil into Prince William Sound, causing significant environmental damage and affecting wildlife populations for decades.
  * **Relevance:** High.
  * **Harm Caused:** Significant locally.
  * **Result:** The Oil Pollution Act of 1990 phased in double hulls for oil tankers and banned some tankers from entering Prince William Sound. Rising public concern about climate change and the first IPCC report occur soon after, but seem to be mostly unrelated.((Nina Joung. //The 1989 Exxon Valdez oil spill, and how we see climate change then and now.// PBS. (2019) [[https://www.pbs.org/wnet/peril-and-promise/2019/03/thirty-years-exxon-valdez/]].))

=== Extreme Weather Events ===

  * **Date:** Various.
  * **Location:** Various.
  * **Description:** Some extreme weather events are more likely to occur as a result of anthropogenic climate change.((//IPCC Sixth Assessment Report.// Ch. 11: Weather and Climate Extreme Events in a Changing Climate. (2021) [[https://www.ipcc.ch/report/ar6/wg1/chapter/chapter-11/]].)) Activists use individual weather events to promote policies designed to reduce climate change.
  * **Relevance:** Moderate.
  * **Harm Caused:** Significant to catastrophic locally.
  * **Result:** There is some evidence that people become more concerned about climate change in response to an extreme weather event, although the evidence is mixed.((Bergquist, Nilsson, and Schultz. //Experiencing a Severe Weather Event Increases Concern About Climate Change.// Frontiers in Psychology **10**. (2019) [[https://www.frontiersin.org/articles/10.3389/fpsyg.2019.00220/full]].)) The impact of extreme weather events on climate policy is even less clear.((Sam Rowen. //Extreme weather and climate policy.// Environmental Politics **32**. (2022) p. 684-707. [[https://www.tandfonline.com/doi/full/10.1080/09644016.2022.2127478]].))

==== Chlorofluorocarbons (CFCs) ====

=== Discovery of the Ozone Hole ===

  * **Date:** 1985.
  * **Location:** Southern Hemisphere.
  * **Description:** CFCs in the stratosphere deplete ozone, especially around Antarctica, allowing more UV light to reach the surface, which causes skin cancers.((//Ozone hole facts.// NASA Ozone Watch. (Accessed November 16, 2023) [[https://ozonewatch.gsfc.nasa.gov/facts/hole_SH.html]].)) This had been predicted and one location in Antarctica had measured declining ozone. Aircraft measurements showed that this was widespread and associated with CFCs.
  * **Relevance:** High.
  * **Harm Caused:** Significant regionally.(("A comprehensive report by United Nations Environment Programme estimated an additional burden of 4500 melanoma cases and 300,000 non-melanoma cases if there is a 10% decrease in the ozone layer." \\
Umar & Tasduq. //Ozone Layer Depletion and Emerging Public Health Concerns.// Frontiers in Oncology **12**. (2022) [[https://www.frontiersin.org/journals/oncology/articles/10.3389/fonc.2022.866733/full]].))
  * **Result:** Vienna Convention of 1985 and Montreal Protocol of 1987, a global treaty which created a phase-out management plan for CFCs.((//Montreal Protocol on Substances that Deplete the Ozone Layer (with annex).// (1987) [[https://treaties.un.org/doc/publication/unts/volume%201522/volume-1522-i-26369-english.pdf]].))

==== Genetically Modified Organisms (GMOs) ====

=== Mad Cow Disease ===

  * **Date:** 1995.
  * **Location:** England, UK.
  * **Description:** Prion disease which spread as cattle were fed meat-and-bone meal, in cattle since 1985 and in humans since 1995.((John Pattison. //The emergence of bovine spongiform encephalopathy and related diseases.// Emerging Infectious Diseases **4.3**. (1998) p. 390-394. [[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2640268/pdf/9716952.pdf]].))
  * **Relevance:** Low.
  * **Harm Caused:** Significant regionally.
  * **Result:** Opposition to GMOs rose by double digits from 1996-1999 in many European countries. Corporate campaigns convinced supermarkets to remove GMOs. National GMO bans led to a de facto GMO moratorium in the EU, which persists today.((Charlie Harrison. //Go Mobilize? Lessons from GM Protests for Pausing AI.// EA Forum. (2023) [[https://forum.effectivealtruism.org/posts/6jxrzk99eEjsBxoMA/go-mobilize-lessons-from-gm-protests-for-pausing-ai#3__Brief_History_of_GM_Crops]].))

=== Cloning of Dolly the Sheep ===

  * **Date:** 1997.
  * **Location:** Scotland, UK.
  * **Description:** Dolly was the first mammal cloned from an adult cell.((//The Life of Dolly.// The Roslin Institute. (Accessed November 16, 2023) [[https://dolly.roslin.ed.ac.uk/facts/the-life-of-dolly/index.html]].))
  * **Relevance:** Low.
  * **Harm Caused:** None.
  * **Result:** Opposition to GMOs rose by double digits from 1996-1999 in many European countries. Corporate campaigns convinced supermarkets to remove GMOs. National GMO bans led to a de facto GMO moratorium in the EU, which persists today.((Charlie Harrison. //Go Mobilize? Lessons from GM Protests for Pausing AI.// EA Forum. (2023) [[https://forum.effectivealtruism.org/posts/6jxrzk99eEjsBxoMA/go-mobilize-lessons-from-gm-protests-for-pausing-ai#3__Brief_History_of_GM_Crops]].))

==== Medical Research Involving Viruses ====

=== COVID-19 Lab Leak Hypothesis ===

  * **Date:** 2019.
  * **Location:** Wuhan, China.
  * **Description:** A new respiratory virus caused a global pandemic, killing millions of people. The virus might have originated from the Wuhan Institute of Virology.
  * **Relevance:** Unclear.
  * **Harm Caused:** Significant globally.
  * **Result:** Some efforts to ban gain-of-function research, but no significant policy changes.((Nate Soares. //Warning Shots Probably Won’t Change The Picture Much.// LessWrong. (2022) [[https://www.lesswrong.com/posts/idipkijjz5PoxAwju/warning-shots-probably-wouldn-t-change-the-picture-much]].))

==== Medical Research Involving Human Subjects ====

//[[https://aiimpacts.org/wp-content/uploads/2023/09/Standards-Case-Study-Institutional-Review-Boards.pdf|Case Study]]//

=== Jewish Chronic Disease Study ===

  * **Date:** 1964.
  * **Location:** New York, USA.
  * **Description:** Elderly patients were injected with live cancer cells without their consent to study how and whether their body would reject them.
  * **Relevance:** High.
  * **Harm Caused:** Low to significant locally.
  * **Result:** Congressional hearings and lawsuits, although the NIH managed to avoid responsibility. Clinical Research Committees, like the one at the NIH's hospital in Bethesda, Maryland, were created at more research hospitals.((Laura Stark. //Protections for Human Subjects in Research: Old Models, New Needs?// MIT Case Studies
in Social and Ethical Responsibilities of Computing. (2022)
[[https://mit-serc.pubpub.org/pub/protections-for-human-subjects/release/1]]. \\
Citing: Laura Stark. //Behind Closed Doors: IRBs and the Making of Ethical Research.// (Chicago, 2012) ))

=== Willowbrook Hepatitis Study ===

  * **Date:** 1971.
  * **Location:** New York, USA.
  * **Description:** Mentally ill children were deliberately infected with hepatitis to study how hepatitis spread at their school.((Philip A. Pecorino. //Case: Willowbrook Experiment.// The City University of New York. (2002) [[https://www.qcc.cuny.edu/socialSciences/ppecorino/MEDICAL_ETHICS_TEXT/Chapter_7_Human_Experimentation/Case_Study_Willowbrook_Experiments.htm]].))
  * **Relevance:** High.
  * **Harm Caused:** Low to significant locally.
  * **Result:** National Research Act of 1974, leading to the Belmont Report of 1978 and the creation of Institutional Review Boards for all research involving human subjects at institutions which receive funding from the federal government.((The National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. //The Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research.// (1978) [[https://www.hhs.gov/ohrp/regulations-and-policy/belmont-report/read-the-belmont-report/index.html]].))

=== Tuskegee Syphilis Study ===

  * **Date:** 1972.
  * **Location:** Alabama, USA.
  * **Description:** Hundreds of black men with syphilis were not told they had the disease for decades and were not given treatment, even after it became available, to "determine through autopsies what damage untreated syphilis does to the human body."((Jean Heller. //AP WAS THERE: Black men untreated in Tuskegee Syphilis Study.// AP News. (2017) [[https://apnews.com/article/business-science-health-race-and-ethnicity-syphilis-e9dd07eaa4e74052878a68132cd3803a]].))
  * **Relevance:** High.
  * **Harm Caused:** Significant locally.
  * **Result:** National Research Act of 1974, leading to the Belmont Report of 1978 and the creation of Institutional Review Boards for all research involving human subjects at institutions which receive funding from the federal government.((The National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. //The Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research.// (1978) [[https://www.hhs.gov/ohrp/regulations-and-policy/belmont-report/read-the-belmont-report/index.html]].))


//Primary author: Jeffrey Heninger.//
AI Impacts Wiki

User Tools

Site Tools

Page Tools
