Moral Uncertainty and the Problem of Fanaticism
Jazon Szabo, Natalia Criado, Jose Such, Sanjay Modgil
Abstract
While there is universal agreement that agents ought to act ethically, there is no agreement as to what constitutes ethical behaviour. To address this problem, recent philosophical approaches to moral uncertainty' propose aggregation of multiple ethical theories to guide agent behaviour. However, one of the foundational proposals for aggregation - Maximising Expected Choiceworthiness (MEC) - has been criticised as being vulnerable to fanaticism; the problem of an ethical theory dominating agent behaviour despite low credence (confidence) in said theory. Fanaticism thus undermines the democratic' motivation for accommodating multiple ethical perspectives. The problem of fanaticism has not yet been mathematically defined. Representing moral uncertainty as an instance of social welfare aggregation, this paper contributes to the field of moral uncertainty by 1) formalising the problem of fanaticism as a property of social welfare functionals and 2) providing non-fanatical alternatives to MEC, i.e. Highest k-trimmed Mean and Highest Median.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 27d9b6ff-7df2-44e4-a19f-c41881a5b5d4Builds on1
Related papers
- Strategyproof Mean Estimation from Multiple-Choice QuestionsAnson Kahng, Gregory Kehne, Ariel D. ProcacciaICML 2020 · 2 citations
- Ethical Dilemmas in Strategic GamesPavel Naumov, Rui-Jie YewAAAI 2021 · 6 citations
- Modeling Voters in Multi-Winner Approval VotingJaelle Scheuerman, Jason L. Harman, Nicholas Mattei, K. Brent VenableAAAI 2021 · 4 citations
- Uncommon Belief in RationalityQi Shi, Pavel NaumovAAAI 2025
- Most Expected Winner: An Interpretation of Winners over Uncertain Voter PreferencesHaoyue Ping, Julia StoyanovichSIGMOD 2023 · 1 citation
