How Humans and LLMs Organize Conceptual Knowledge: Exploring Subordinate Categories in Italian
Andrea Pedrotti, Giulia Rambelli, Caterina Villani, Marianna Bolognesi
Abstract
People can categorize the same entity at multiple taxonomic levels, such as basic (bear), superordinate (animal), and subordinate (grizzly bear). While prior research has focused on basic-level categories, this study is the first attempt to examine the organization of categories by analyzing exemplars produced at the subordinate level. We present a new Italian psycholinguistic dataset of human-generated exemplars for 187 concrete words. We then use these data to evaluate whether textual and vision LLMs produce meaningful exemplars that align with human category organization across three key tasks: exemplar generation, category induction, and typicality judgment. Our findings show a low alignment between humans and LLMs, consistent with previous studies. However, their performance varies notably across different semantic domains. Ultimately, this study highlights both the promises and the constraints of using AI-generated exemplars to support psychological and linguistic research. 1
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on5
- Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
- Perceiver: General Perception with Iterative AttentionAndrew Jaegle, Felix Gimeno, Andy Brock, Oriol Vinyals et al.ICML 2021 · 1,399 citations
- Climbing towards NLU: On Meaning, Form, and Understanding in the Age of DataEmily M. Bender, Alexander KollerACL 2020 · 914 citations
- What matters when building vision-language models?Hugo Laurençon, Léo Tronchon, Matthieu Cord, Victor SanhNeurIPS 2024 · 401 citations
- Connecting the Dots: Evaluating Abstract Reasoning Capabilities of LLMs Using the New York Times Connections Word GamePrisha Samadarshi, Mariam Mustafa, Anushka Kulkarni, Raven Rothkopf et al.EMNLP 2024 · 5 citations
Related papers
- Seeing Through Words, Speaking Through Pixels: Deep Representational Alignment Between Vision and Language ModelsZoe Wanying He, Sean Trott, Meenakshi KhoslaEMNLP 2025 · 2 citations
- Grounding Visual Illusions in Language: Do Vision-Language Models Perceive Illusions Like Humans?Yichi Zhang, Jiayi Pan, Yuchen Zhou, Rui Pan et al.EMNLP 2023 · 9 citations
- Unveiling Multi-level and Multi-modal Semantic Representations in the Human Brain using Large Language ModelsYuko Nakagi, Takuya Matsuyama, Naoko Koide-Majima, Hiroto Yamaguchi et al.EMNLP 2024 · 7 citations
- Language-biased image classification: evaluation based on semantic representationsYoann Lemesle, Masataka Sawayama, Guillermo Valle Pérez, Maxime Adolphe et al.ICLR 2022 · 8 citations
- Conceptual structure coheres in human cognition but not in large language modelsSiddharth Suresh, Kushin Mukherjee, Xizheng Yu, Wei-Chun Huang et al.EMNLP 2023 · 7 citations
