Beneficial AI and Machine Ethics

To ensure that AI systems have a positive impact, it is not enough to focus on mitigating risks of harmful behaviour. Rather, we need to instill beneficial goals in AIs and ensure that these align with our values and ethical principles. However, there are many potential frameworks to be considered for this purpose.


Defining the principles that advanced AI systems should follow is no trivial task. There are many plausible candidates, such as complying with users' preferences, following the law, or doing what is ethical. While each of these proposals has its attractions, they all face limitations as well, and may come into conflict with each other. We explore these questions and consider what it would mean to design an AI system that promotes the wellbeing of users and of society as a whole. We also discuss how AI systems should deal with situations where there is uncertainty about the right course of action.

Further reading

W. Wallach and C. Allen, Moral Machines: Teaching Right From Wrong. Oxford University Press, 2008.

J. Nay, "Law Informs Code," 2023. [Online]. Available:

S. Barocas, M. Hardt, and A. Narayanan, Fairness and Machine Learning: Limitations and Opportunities. 2023. [Online]. Available:

R. Layard and J. De Neve, Wellbeing: Science and Policy. Cambridge University Press, India, 2023.

M. J. Sandel, What Money Can't Buy: The Moral Limits of Markets. Farrar, Straus and Giroux, 2012.

D. Hendrycks et al., "What Would Jiminy Cricket Do? Towards Agents That Behave Morally," 2022. [Online]. Available:

M. D. Adler, "Introduction," in Measuring Social Welfare: An Introduction. Oxford Academic, New York, 2019. [Online]. Available:

S. Lukes, Moral Relativism. 2008.

K. de Lazari-Radek and P. Singer, Utilitarianism: A Very Short Introduction, Very Short Introductions. Oxford Academic, Oxford, 2017. [Online]. Available:

S. Kagan, Normative Ethics. Westview Press, 1998.

S. Newberry and T. Ord, "The Parliamentary Approach to Moral Uncertainty," 2022. [Online]. Available:

S. Darwall, Deontology: A Very Short Introduction. Oxford University Press, 2013.

Discussion Questions

Review Questions