Lune

ACL Findings2025

Question Answering in Climate Adaptation for Agriculture: Model Development and Evaluation with Expert Feedback

Vincent Nguyen, Sarvnaz Karimi, Willow Hallgren, Mahesh Prakash

Abstract

The generative capabilities of the large language models (LLMs) are deployed for domainspecific question answering systems. However, their ability to answer climate adaptation questions remains unclear. In particular, can they be used by agronomists and climate scientists to answer questions on the best climate adaptation strategies? Answering questions in this domain requires knowledge of climate data and its uncertainties, and the ability to link them to the broader climate literature while accommodating the unique constraints of users and experts. We investigate the generative and evaluative capabilities of several state-of-the-art LLMs, open-source and proprietary, on climate adaptation for agriculture questions posed by domain experts using evaluation criteria designed by the experts.

We propose an iterative exploration framework that enables LLMs to dynamically aggregate information from heterogeneous sources, such as text from climate literature and structured tabular climate data from climate model projections and historical observations. Our experiments demonstrate that LLMs can aggregate heterogeneous data to ( 1) answer questions, but at a trade-off between presentation quality and epistemological accuracy; and, (2) evaluate answers, but are not as competent at identifying high-quality answers and erroneous information compared to domain experts.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • G-Eval: NLG Evaluation using Gpt-4 with Better Human Alignment cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on15

  1. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  2. G-Eval: NLG Evaluation using Gpt-4 with Better Human AlignmentYang Liu, Dan Iter, Yichong Xu, Shuohang Wang et al.EMNLP 2023 · 549 citations
  3. S2ORC: The Semantic Scholar Open Research CorpusKyle Lo, Lucy Lu Wang, Mark Neumann, Rodney Kinney et al.ACL 2020 · 424 citations
  4. The Unreliability of Explanations in Few-shot Prompting for Textual ReasoningXi Ye, Greg DurrettNeurIPS 2022 · 272 citations
  5. Can Large Language Models Be an Alternative to Human Evaluations?David Cheng-Han Chiang, Hung-yi LeeACL 2023 · 254 citations

Related papers

  1. ClimaQA: An Automated Evaluation Framework for Climate Question Answering ModelsVeeramakali Vignesh Manivannan, Yasaman Jafari, Srikar Eranky, Spencer Ho et al.ICLR 2025
  2. GCA Framework: A GCC Countries-Grounded Dataset and Agentic Pipeline for Climate Decision SupportMuhammad Umer Sheikh, Khawar Shehzad, Salman Khan, Fahad Shahbaz Khan et al.ACL 2026 · 1 citation
  3. Human-LLM Hybrid Text Answer Aggregation for Crowd AnnotationsJiyi LiEMNLP 2024 · 1 citation
  4. AgriEval: A Comprehensive Chinese Agricultural Benchmark for Large Language ModelsLian Yan, Haotian Wang, Chen Tang, Haifeng Liu et al.AAAI 2026 · 3 citations
  5. Assessing Large Language Models on Climate InformationJannis Bulian, Mike S. Schäfer, Afra Amini, Heidi Lam et al.ICML 2024 · 39 citations
Dusk over the sea between two cliffs drawn in fine vertical lines