Take Action
Course
Curriculum
Download Textbook
1. Overview of Catastrophic AI Risks
0.1
Preface
1.1
Overview of Catastrophic AI Risks
1.2
Malicious Use
1.3
AI Race
1.4
Organizational Risks
1.5
Rogue AIs
1.6
Discussion of Connections Between Risks
2. AI Fundamentals
2.1
AI Fundamentals
2.2
Artificial Intelligence & Machine Learning
2.3
Deep Learning
2.4
Scaling Laws
2.5
Conclusion
3. Single Agent Safety
3.1
Single Agent Safety
3.2
Monitoring
3.3
Robustness
3.4
Alignment
3.5
Systemic Safety
3.6
Safety and General Capabilities
3.7
Conclusion
4. Safety Engineering
4.1
Safety Engineering
4.2
Risk Decomposition
4.3
Nines of Reliability
4.4
Safe Design Principles
4.5
Component Failure Accident Models and Methods
4.6
Systemic Factors
4.7
Tail Events and Black Swans
4.8
Conclusion
5. Complex Systems
5.1
Complex Systems
5.2
Introduction to Complex Systems
5.3
Complex Systems for AI Safety
5.4
Conclusion
6. Beneficial AI and Machine Ethics
6.1
Beneficial AI and Machine Ethics
6.2
Law
6.3
Fairness
6.4
The Economic Engine
6.5
Wellbeing
6.6
Preferences
6.7
Happiness
6.8
Social Welfare Functions
6.9
Moral Uncertainty
7. Collective Action Problems
7.1
Collective Action Problems
7.2
Game Theory
7.3
Cooperation
7.4
Conflict
7.5
Evolutionary Pressures
7.6
Conclusion
8. Governance
8.1
Governance
8.2
Timelines
8.3
Distribution
8.4
Compute Governance
8.5
Corporate Governance
8.6
National Governance
8.7
International Governance
8.8
Conclusion
9. Appendices
9.1
Acknowledgements
10.1
App. A: Normative Ethics
11.1
App. B: Utility Functions
12.1
App. C: Reinforcement Learning
13.1
App. D: Long-Tailed and Thin-Tailed Distributions
14.1
App. E: Evolutionary Game Theory
15.1
App. F: Other Cooperation Mechanisms
16.1
App. G: Intrasystem Conflict Causes
All
Section
Appendix
2.1

AI Fundamentals

To understand the risks that artificial intelligence poses and to learn what measures we can take to mitigate them, it is essential to understand the technology itself: how it works, how it is used, and where its strengths and weaknesses lie.

Summary

To understand the risks that artificial intelligence (AI) poses and to learn what measures we can take to mitigate them, it is essential to understand the technology itself: how it works, how it is used, and where its strengths and weaknesses lie. We describe key concepts in machine learning (ML), the approach that powers most modern AI systems, with a particular focus on the technologies that power Large Language Models such as OpenAI's GPT-4 or Google's Gemini. We introduce the phenomenon of scaling laws, where AI systems' performance improves predictably as the amounts of data and computation used during their training are increased, and consider what this implies for future AI progress.

Further reading

Introductory resources on neural networks, LLMs and AI scaling trends:
‍
"But what is a Neural Network". 3Blue1Brown, 2017. url: https://www.3blue1brown.com/lessons/neural-networks
A. Karpathy, "Intro to Large Language Models".  2023. url: https://www.youtube.com/watch?v=zjkBMFhNj_g
Epoch AI. "Key Trends and Figures in Machine Learning". 2024 url: https://epochai.org/trends
‍
Reference works for advanced readers:
‍
I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. MIT Press, 2016.

S. Russell, P. Norvig, and E. Davis, Artificial Intelligence: A Modern Approach, 3rd ed. Upper Saddle River, NJ: Prentice Hall, 2010.

J. Hoffman et al., "Training Compute-Optimal Large Language Models," DeepMind, 2022.

R. S. Sutton and A. Barto, Reinforcement Learning: An Introduction. The MIT Press, 2018.

Discussion Questions

Review Questions

0.1
1.1
1.2
1.3
1.4
1.5
1.6
2.1
2.2
2.3
2.4
2.5
3.1
3.2
3.3
3.4
3.5
3.6
3.7
4.1
4.2
4.3
4.4
4.5
4.6
4.7
4.8
5.1
5.2
5.3
5.4
6.1
6.2
6.3
6.4
6.5
6.6
6.7
6.8
6.9
7.1
7.2
7.3
7.4
7.5
7.6
8.1
8.2
8.3
8.4
8.5
8.6
8.7
8.8
9.1
10.1
11.1
12.1
13.1
14.1
15.1
16.1
Get Involved
CourseFacilitateTake ActionSubmit Feedback
Textbook Resources
Table of ContentsDownload TextbookContent for Curriculum
Terms of ServicePrivacy Policy

Cookies Notice: This website uses cookies to identify pages that are being used most frequently. This helps us analyze data about web page traffic and improve our website. We only use this information for the purpose of statistical analysis and then the data is removed from the system. We do not and will never sell user data. Read more about our cookie policy on our privacy policy. Please contact us if you have any questions.

© 2024 Center for AI Safety
Built by Osborn Design Works