Lune

EMNLP2021Top-tier venue

Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequences

Denis Emelin, Ronan Le Bras, Jena D. Hwang, Maxwell Forbes, Yejin Choi

Abstract

In social settings, much of human behavior is governed by unspoken rules of conduct rooted in societal norms. For artificial systems to be fully integrated into social environments, adherence to such norms is a central prerequisite. To investigate whether language generation models can serve as behavioral priors for systems deployed in social settings, we evaluate their ability to generate action descriptions that achieve predefined goals under normative constraints. Moreover, we examine if models can anticipate likely consequences of actions that either observe or violate known norms, or explain why certain actions are preferable by generating relevant norm hypotheses. For this purpose, we introduce Moral Stories, a crowd-sourced dataset of structured, branching narratives for the study of grounded, goaloriented social reasoning. Finally, we propose decoding strategies that combine multiple expert models to significantly improve the quality of generated actions, consequences, and norms compared to strong baselines. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

34 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Evaluating the Moral Beliefs Encoded in LLMs cites Aligning AI With Shared Human Values
  • Evaluating the Moral Beliefs Encoded in LLMs cites When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment
  • Abductive Commonsense Reasoning cites WinoGrande: An Adversarial Winograd Schema Challenge at Scale
  • Training Socially Aligned Language Models on Simulated Social Interactions cites Aligning AI With Shared Human Values
  • Training Socially Aligned Language Models on Simulated Social Interactions cites The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems
  • The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems cites Aligning AI With Shared Human Values
  • The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems cites The Curious Case of Neural Text Degeneration
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers34

  1. Evaluating the Moral Beliefs Encoded in LLMsNino Scherrer, Claudia Shi, Amir Feder, David M. BleiNeurIPS 2023 · 316 citations
  2. When to Make Exceptions: Exploring Language Models as Accounts of Human Moral JudgmentZhijing Jin, Sydney Levine, Fernando Gonzalez Adauto, Ojasv Kamal et al.NeurIPS 2022 · 146 citations
  3. The Moral Integrity Corpus: A Benchmark for Ethical Dialogue SystemsCaleb Ziems, Jane A. Yu, Yi-Chia Wang, Alon Y. Halevy et al.ACL 2022 · 127 citations
  4. From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP ModelsShangbin Feng, Chan Young Park, Yuhan Liu, Yulia TsvetkovACL 2023 · 117 citations
  5. Training Socially Aligned Language Models on Simulated Social InteractionsRuibo Liu, Ruixin Yang, Chenyan Jia, Ge Zhang et al.ICLR 2024 · 97 citations

Builds on11

  1. The Curious Case of Neural Text DegenerationAri Holtzman, Jan Buys, Li Du, Maxwell Forbes et al.ICLR 2020 · 4,112 citations
  2. WinoGrande: An Adversarial Winograd Schema Challenge at ScaleKeisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, Yejin ChoiAAAI 2020 · 3,037 citations
  3. Aligning AI With Shared Human ValuesDan Hendrycks, Collin Burns, Steven Basart, Andrew Critch et al.ICLR 2021 · 878 citations
  4. Abductive Commonsense ReasoningChandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi et al.ICLR 2020 · 521 citations
  5. (Comet-) Atomic 2020: On Symbolic and Neural Commonsense Knowledge GraphsJena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Jeff Da et al.AAAI 2021 · 458 citations

Related papers

  1. Contextualizing Language Models for Norms Diverging from Social MajorityNiklas Kiehne, Hermann Kroll, Wolf-Tilo BalkeEMNLP Findings 2022 · 2 citations
  2. SCRUPLES: A Corpus of Community Ethical Judgments on 32, 000 Real-Life AnecdotesNicholas Lourie, Ronan Le Bras, Yejin ChoiAAAI 2021 · 147 citations
  3. MoCa: Measuring Human-Language Model Alignment on Causal and Moral Judgment TasksAllen Nie, Yuhui Zhang, Atharva Amdekar, Chris Piech et al.NeurIPS 2023 · 78 citations
  4. Tales of Morality: Comparing Human- and LLM-Generated Moral Stories from Visual CuesRezvaneh Rezapour, Sullam Jeoung, Zhiwen You, Jana DiesnerEMNLP Findings 2025 · 2 citations
  5. Social Chemistry 101: Learning to Reason about Social and Moral NormsMaxwell Forbes, Jena D. Hwang, Vered Shwartz, Maarten Sap et al.EMNLP 2020 · 11 citations
Dusk over the sea between two cliffs drawn in fine vertical lines