Take Action
Course
Curriculum
Download Textbook
1. Overview of Catastrophic AI Risks
0.1
Preface
1.1
Overview of Catastrophic AI Risks
1.2
Malicious Use
1.3
AI Race
1.4
Organizational Risks
1.5
Rogue AIs
1.6
Discussion of Connections Between Risks
2. AI Fundamentals
2.1
AI Fundamentals
2.2
Artificial Intelligence & Machine Learning
2.3
Deep Learning
2.4
Scaling Laws
2.5
Conclusion
3. Single Agent Safety
3.1
Single Agent Safety
3.2
Monitoring
3.3
Robustness
3.4
Alignment
3.5
Systemic Safety
3.6
Safety and General Capabilities
3.7
Conclusion
4. Safety Engineering
4.1
Safety Engineering
4.2
Risk Decomposition
4.3
Nines of Reliability
4.4
Safe Design Principles
4.5
Component Failure Accident Models and Methods
4.6
Systemic Factors
4.7
Tail Events and Black Swans
4.8
Conclusion
5. Complex Systems
5.1
Complex Systems
5.2
Introduction to Complex Systems
5.3
Complex Systems for AI Safety
5.4
Conclusion
6. Beneficial AI and Machine Ethics
6.1
Beneficial AI and Machine Ethics
6.2
Law
6.3
Fairness
6.4
The Economic Engine
6.5
Wellbeing
6.6
Preferences
6.7
Happiness
6.8
Social Welfare Functions
6.9
Moral Uncertainty
7. Collective Action Problems
7.1
Collective Action Problems
7.2
Game Theory
7.3
Cooperation
7.4
Conflict
7.5
Evolutionary Pressures
7.6
Conclusion
8. Governance
8.1
Governance
8.2
Timelines
8.3
Distribution
8.4
Compute Governance
8.5
Corporate Governance
8.6
National Governance
8.7
International Governance
8.8
Conclusion
9. Appendices
9.1
Acknowledgements
10.1
App. A: Normative Ethics
11.1
App. B: Utility Functions
12.1
App. C: Reinforcement Learning
13.1
App. D: Long-Tailed and Thin-Tailed Distributions
14.1
App. E: Evolutionary Game Theory
15.1
App. F: Other Cooperation Mechanisms
16.1
App. G: Intrasystem Conflict Causes
All
Section
Appendix
8.3

Distribution

In this section we discuss three main dimensions of how aspects of AI systems are distributed.

No items found.

Review Questions

What is an AI singleton? What factors make AI singleton scenarios more or less likely?

‍

‍

Answer:

An AI singleton is an AI that gains decisive and lasting power over all other AIs and humans. Factors making it more likely include a monopoly over AI development, a fast takeoff, and incentives to capture power. Decentralized development makes it less likely.

View Answer
Hide Answer

What are some risks from (a) highly open and widely available AI systems, and (b)from tight control over AI by a narrow group?

Answer:

More open AI models increase risks of bottom-up misuse, where members of the general public leverage the technology to pursue negative outcomes. The more people who can access advanced AIs, the more likely someone will use them harmfully.

More tightly controlled powerful AIs increase risks of top-down misuse by government officials and elites. AIs controlled by a small group could be used to lock in their values without considering humanity's interests broadly.

‍

View Answer
Hide Answer
0.1
1.1
1.2
1.3
1.4
1.5
1.6
2.1
2.2
2.3
2.4
2.5
3.1
3.2
3.3
3.4
3.5
3.6
3.7
4.1
4.2
4.3
4.4
4.5
4.6
4.7
4.8
5.1
5.2
5.3
5.4
6.1
6.2
6.3
6.4
6.5
6.6
6.7
6.8
6.9
7.1
7.2
7.3
7.4
7.5
7.6
8.1
8.2
8.3
8.4
8.5
8.6
8.7
8.8
9.1
10.1
11.1
12.1
13.1
14.1
15.1
16.1
Get Involved
CourseFacilitateTake ActionSubmit Feedback
Textbook Resources
Table of ContentsDownload TextbookContent for Curriculum
Terms of ServicePrivacy Policy

Cookies Notice: This website uses cookies to identify pages that are being used most frequently. This helps us analyze data about web page traffic and improve our website. We only use this information for the purpose of statistical analysis and then the data is removed from the system. We do not and will never sell user data. Read more about our cookie policy on our privacy policy. Please contact us if you have any questions.

© 2024 Center for AI Safety
Built by Osborn Design Works