AI Impacts Wiki research_problems

AI Impacts key questions of interest

2022-09-21T07:37:41+00:00

@@ -1 +1,60 @@
+ ====== AI Impacts key questions of interest ======
+ 
+ // Published 17 December, 2020 //
+ 
+ 
+ 
+ <HTML>
+ <p>This is a list of questions that AI Impacts focuses on answering.</p>
+ </HTML>
+ 
+ 
+ ===== Details =====
+ 
+ 
+ <HTML>
+ <p>We are interested in understanding how the development of advanced AI will proceed and how it may affect humanity, especially insofar as these are relevant to efforts to improve the outcomes. This is a list of questions within this topic that we currently consider particularly important to answer.</p>
+ </HTML>
+ 
+ 
+ ==== List ====
+ 
+ 
+ <HTML>
+ <ul>
+ <li><div class="li"><strong>AI RISK:</strong> What type and degree of risk will be posed to humanity by advanced AI systems?</div></li>
+ <li><div class="li">
+ <strong>CHARACTER:</strong> What will early advanced AI systems be like?
+                   <ul>
+ <li><div class="li"><strong>ARCHITECTURE:</strong> What types of algorithms will advanced AI systems use?</div></li>
+ <li><div class="li"><strong>AGENCY:</strong> Will the bulk of advanced AI systems be in the form of ‘agents’? (If so, in what sense? Will they pursue ‘goals’? Can we say anything about the nature the goals or the pursuit?)</div></li>
+ <li><div class="li">
+ <strong>PRICE:</strong> How much will the first <a href="/doku.php?id=clarifying_concepts:human-level_ai">human-level AI</a> systems cost?
+                     </div></li>
+ </ul>
+ </div></li>
+ <li><div class="li">
+ <strong>TIMELINES:</strong> When will <a href="/doku.php?id=clarifying_concepts:human-level_ai">human-level AI</a> be developed? (When will other important AI milestones take place?)
+                 </div></li>
+ <li><div class="li">
+ <strong>TAKE-OFF SPEED:</strong> How rapid is the development of AI likely to be near human-level?
+                   <ul>
+ <li><div class="li"><strong>DISCONTINUITY:</strong> Will there be abrupt progress in AI development at around human-level performance?</div></li>
+ <li><div class="li"><strong>INTELLIGENCE EXPLOSION:</strong> How much will AI development be accelerated by feedback from AI-based automation of the process?</div></li>
+ </ul>
+ </div></li>
+ <li><div class="li">
+ <strong>PRE-AI DEVELOPMENTS:</strong> What developments will take place before advanced AI is developed?
+                   <ul>
+ <li><div class="li"><strong>PATHS TO HLAI:</strong> By what methods is advanced AI likely to come about? (e.g. will human-level AI be developed via brain emulation before it is developed via machine learning? Will neuroscientific understanding play a large role in development?)</div></li>
+ <li><div class="li"><strong>CONTEMPORARY EVENTS:</strong> How will the world be relevantly different at the time that advanced AI is developed?</div></li>
+ <li><div class="li"><strong>WARNING SIGNS:</strong> Should we expect advance notice of disruptive change from AI? (What would it look like?)</div></li>
+ </ul>
+ </div></li>
+ <li><div class="li"><strong>POST-AI SOCIETY:</strong> If advanced AI transforms the world, what will the world look like afterwards? (e.g. What will the economic impacts be? Will humans flourish? What roles will AI systems play in society?)</div></li>
+ <li><div class="li"><strong>ACTIONS:</strong> What can we say about the impact of contemporary choices on long-term outcomes?</div></li>
+ </ul>
+ </HTML>
+ 
+

Possible Empirical Investigations

2022-09-21T07:37:41+00:00

@@ -1 +1,39 @@
+ ====== Possible Empirical Investigations ======
+ 
+ // Published 25 February, 2015; last updated 12 January, 2021 //
+ 
+ <HTML>
+ <p>In the course of our work, we have noticed a number of empirical questions which bear on our forecasts and might be (relatively) cheap to resolve. In the future we hope to address some of these.</p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <ul>
+ <li><div class="li">
+ <a href="/doku.php?id=featured_articles:research_topic_hardware_software_and_ai" title="Research topic: Hardware, software and AI">Our partial list of investigations into forecasting AI timelines</a>
+ </div></li>
+ <li><div class="li">
+ <a href="/doku.php?id=featured_articles:list_of_multipolar_research_projects" title="List of multipolar research projects">Our list of investigations that bear on ‘multipolar’ AI scenarios</a>
+ </div></li>
+ <li><div class="li">Look at the work of ancient or enlightenment mathematicians and control for possible selection effects in <a href="https://sites.google.com/site/aiimpactslibrary/resolutions-of-mathematical-conjectures">this analysis</a> of historical mathematical conjectures.
+                 </div></li>
+ <li><div class="li">Look for historical characterizations of the AI problem, and try to obtain unbiased (though uninformed) breakdowns of the problem which could be used to gauge progress.</div></li>
+ <li><div class="li">Identify previous examples of technological projects with clear long-term goals, and then produce estimates of the time required to achieve those goals to varying degrees.</div></li>
+ <li><div class="li">Analyze the performance of different versions of software for benchmark problems, like SAT solving or chess, and determine the extent to which hardware and software progress facilitated improvement.</div></li>
+ <li><div class="li">Obtain a clearer picture of the extent to which historical developments in neuroscience have played a meaningful role in historical progress in AI. Our impression is that this influence has been minimal, but this judgment might be attributable to hindsight bias.</div></li>
+ <li><div class="li">In the field of AI, estimate the ratio of spending on hardware to spending on researchers.</div></li>
+ <li><div class="li">Estimate the change in inputs in mathematicians, scientists, or engineers, as a complement to estimates for rates of progress in those fields.</div></li>
+ <li><div class="li">Estimate the historical and present size of the AI field, ideally with plausible adjustments for quality (for example performing in-depth investigations for a small number of random samples, perhaps invoking expert opinion) and using these as a basis for quality-adjustments.</div></li>
+ <li><div class="li">
+ <a href="http://lukemuehlhauser.com/some-studies-which-could-improve-our-strategic-picture-of-superintelligence/">Luke Muehlhauser</a> and the <a href="http://futureoflife.org/static/data/documents/research_survey.pdf">Future of Life Institute</a>‘s section on forecasting both list further projects.
+                 </div></li>
+ </ul>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p>Unfortunately this is an incomplete list (even of the ideas which have struck as promising during this project). We are beginning to flesh it out further in our aforementioned <a href="/doku.php?id=featured_articles:research_topic_hardware_software_and_ai" title="Research topic: Hardware, software and AI">list of projects bearing on AI timelines</a>.</p>
+ </HTML>
+ 
+

Promising research projects

2022-09-21T07:37:41+00:00

@@ -1 +1,153 @@
+ ====== Promising research projects ======
+ 
+ // Published 05 April, 2018; last updated 26 March, 2021 //
+ 
+ <HTML>
+ <p>This is an incomplete list of concrete projects that we think are tractable and important. We may do any of them ourselves, but many also seem feasible to work on independently. Those we consider especially well suited to this are marked Ψ. More potential projects are listed <a href="/doku.php?id=research_problems:possible_empirical_investigations">here</a>.</p>
+ </HTML>
+ 
+ 
+ ==== Project ====
+ 
+ 
+ <HTML>
+ <p class="p1"><strong>Review the literature on forecasting (in progress) Ψ</strong></p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p class="p1">Summarize what is known about procedures that produce good forecasts, and measures that are relatively easier to forecast. This may involve reading secondary sources, or collecting past forecasts and investigating what made some of them successful.</p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p>This would be an input to improving our own forecasting practices, and to knowing which other forecasting efforts to trust.</p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p>We have <a href="/doku.php?id=featured_articles:evidence_on_good_forecasting_practices_from_the_good_judgment_project">reviewed</a> some literature associated with the Good Judgment Project in particular.</p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p class="p1"><strong>Review considerations regarding the chance of local, fast takeoff Ψ</strong></p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p class="p1">We have a list of considerations <a href="/doku.php?id=featured_articles:likelihood_of_discontinuous_progress_around_the_development_of_agi">here</a>. If you find local, fast take-off likely, check if the considerations that lead you to this view are represented. Alternately, interview someone else with a strong position about the considerations they find important. If there are any arguments or counterarguments that you think are missing, write a short page explaining the case.</p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p>Collecting arguments on this topic is helpful because opinion among well-informed thinkers on the topic seems to diverge from what would be expected given the arguments that we know about. This suggests that we are missing some important considerations, that we would need to well assess the chance of local, fast takeoff.</p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p><strong>Quantitatively model an intelligence explosion Ψ</strong></p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p>An intelligence explosion (or ‘recursive self-improvement’) consists of a feedback loop where researcher efforts produce scientific progress, which produces improved AI performance, which produces more efficient researcher efforts. This forms a loop, because the researchers involved are artificial themselves.</p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p>Though this loop does not yet exist, relatively close analogues to all of the parts of it already occur: for instance, researcher efforts do lead to scientific progress; scientific progress does lead to better AI; better AI does lead to more capacity at the kinds of tasks that AI can do.</p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p>Collect empirical measurements of proxies like these, for different parts of the hypothesized loop (each part of this could be a stand-alone project). Model the speed of the resulting loop if they were put together, under different background conditions.</p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p>This would give us a very rough estimate of the contribution of intelligence explosion dynamics to the speed of intelligence growth in a transition to an AI-based economy. Also, a more detailed model may inform our understanding of available strategies to improve outcomes.</p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p><strong>Interview AI researchers on topics of interest Ψ</strong></p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p class="p1">Find an AI researcher with views on matters of interest (e.g. AI risk, timelines, the relevance of neuroscience to AI progress) and interview them. Write a summary, or transcript (with their permission). Some examples <a href="/doku.php?id=ai_timelines:conversation_with_steve_potter">here</a>, <a href="/doku.php?id=ai_timelines:conversation_with_tom_griffiths">here</a>, <a href="http://aiimpacts.org/joscha-bach-on-the-unfinished-steps-to-human-level-ai/">here</a>. (If you do not expect to run an interview well enough to make a good impression on the interviewee, consider practicing elsewhere first, so as not to discourage interacting with similar researchers in the future.)</p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p>Talking to AI researchers about their views can be informative about the nature of AI research (e.g. What problems are people trying to solve? <a href="/doku.php?id=ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:2016_expert_survey_on_progress_in_ai">How much does it seem like hardware matters?</a>), and provide an empirically informed take on questions and considerations of interest to us (e.g. <a href="http://aiimpacts.org/what-do-ml-researchers-think-you-are-wrong-about/">Current techniques seem really far from general</a>). They also tell us about state of opinion within the AI research community, which may be relevant in itself.</p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p class="p1"><strong>Review what is known about the relative intelligence of humans, chimps, and other animals (in progress)</strong></p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p>Review efforts to measure animal and human intelligence on a single scale, and efforts to quantify narrower cognitive skills across a range of animals.</p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p>Humans are radically more successful than other animals, in some sense. This is <a href="/doku.php?id=featured_articles:likelihood_of_discontinuous_progress_around_the_development_of_agi">taken as reason</a> to expect that small modifications to brain design (for instance whatever evolution did between the similar brains of chimps and humans) can produce outsized gains in some form of mental performance, and thus that AI researchers may see similar astonishing progress near human-level AI.</p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p>However without defining or quantifying the mental skills of any relevant animals, it is unclear a) whether <em>individual intelligence</em> in particular accounts for humans’ success (rather than e.g. ability to accrue culture and technology), b) whether the gap in capabilities between chimps and humans is larger than expected (maybe chimps are also astonishingly smarter than smaller mammals), or c) whether the success stems from something that evolution was ‘intentionally’ progressing on. These things are all relevant to the strength of an argument for AI ‘fast take-off’ based on human success over chimps (see <a href="/doku.php?id=featured_articles:likelihood_of_discontinuous_progress_around_the_development_of_agi">here</a>).</p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p><strong>Review explanations for humans’ radical success over apes (in progress)</strong></p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p>Investigate what is known about the likely causes of human success, relative to that of other similar animals. In particular, we are interested in how likely improvement in individual cognitive ability is to account for this (as opposed to say communication and group memory abilities).</p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p>This would help resolve the same issues described in the last section (‘Review what is known about the relative intelligence of humans, chimps, and other animals’).</p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p><strong>Collect data on time to cross the human range on intellectual skills where machines have surpassed us (in progress) Ψ</strong></p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p class="p1">For intellectual skills where machines have surpassed humans, find out how long it took to go from the worst performance to average human skill, and from average human skill to superhuman skill.</p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p>This would contribute to <a href="/doku.php?id=speed_of_ai_transition:range_of_human_performance:the_range_of_human_intelligence">this project</a>.</p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p><strong>Measure the importance of hardware progress in a specific narrow AI trajectory Ψ</strong></p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p class="p1">Take an area of AI progress, and assess how much of annual improvement can be attributed to hardware improvements vs. software improvements, or what the more detailed relationship between the two is.</p>
+ </HTML>
+ 
+ 
+ <HTML>
+ <p>Understanding the overall importance of hardware progress and software progress (and other factors) in overall AI progress lets us know to what extent our future expectations should be a function of expected hardware developments, versus software developments. This both alters what our timelines look like (e.g. see <a href="/doku.php?id=ai_timelines:human-level_hardware_timeline">here</a>), and tells us what we should be researching to better understand AI timelines.</p>
+ </HTML>
+ 
+