Lune

EMNLP Findings2025

Automating Alternative Generation in Decision-Making

Yevhen Kostiuk, Clara Seyfried, Chris Reed

Abstract

In decision making, generating alternative solutions is crucial for solving a problem. However, cognitive biases can impede this process by constraining individual decision makers' creativity. To address this issue, we introduce a new task for automatically generating alternatives, inspired by the process of human "brainstorming". We define alternative options based on atomic action components and present a dataset of 106 annotated Reddit r/Advice posts containing unique alternative options extracted from users' replies. We also introduce new metrics to assess the quality of generated components, including distinctiveness, creativity, upvote-weighted, crowd intersection, and final commit intersection scores. As a baseline, we evaluated the large language models (LLMs) LLaMa3:8b, LLaMa3.1:8b, and Gemma 2:9b on the alternative component generation task. On the one hand, models demonstrated high creativity (ability to generate options beyond what Reddit users suggested) and performed well at proposing distinct alternatives. A subset of generated components was manually evaluated and found overall useful. This indicates that LLMs might be used to extend lists of alternative options, helping decision makers consider a problem from different perspectives. On the other hand, LLMs' outputs often failed to align with human suggestions, implying that they still tend to miss important components.

The code and annotation guidelines can be found in the project's GitHub repository 1 . The dataset is available via the link 2 .

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Builds on2

  1. AI-Augmented Brainwriting: Investigating the use of LLMs in group ideationOrit Shaer, Angelora Cooper, Osnat Mokryn, Andrew L. Kun et al.CHI 2024 · 120 citations
  2. SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian LanguagesHoly Lovenia, Rahmad Mahendra, Salsabil Maulana Akbar, Lester James V. Miranda et al.EMNLP 2024 · 8 citations

Related papers

  1. Cognitive Bias in Decision-Making with LLMsJessica Maria Echterhoff, Yao Liu, Abeer Alessa, Julian J. McAuley et al.EMNLP Findings 2024 · 62 citations
  2. Selecting Better Samples from Pre-trained LLMs: A Case Study on Question GenerationXingdi Yuan, Tong Wang, Yen-Hsiang Wang, Emery Fine et al.ACL Findings 2023 · 21 citations
  3. Rubrik's Cube: Testing a New Rubric for Evaluating Explanations on the CUBE datasetDiana Galván-Sosa, Gabrielle Gaudeau, Pride Kavumba, Yunmeng Li et al.ACL 2025
  4. Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent ThinkingHarsh Kumar, Jonathan Vincentius, Ewan Jordan, Ashton AndersonCHI 2025 · 107 citations
  5. Oracle-Guided Program Selection from Large Language ModelsZhiyu Fan, Haifeng Ruan, Sergey Mechtaev, Abhik RoychoudhuryISSTA 2024 · 4 citations
Dusk over the sea between two cliffs drawn in fine vertical lines