Lune

EMNLP Findings2023

Licon: A Diverse, Controllable and Challenging Linguistic Concept Learning Benchmark

Shenglong Yu, Ying Zhang, Wenya Guo, Zhengkun Zhang, Ru Zhou, Xiaojie Yuan

Abstract

Concept Learning requires learning the definition of a general category from given training examples. Most of the existing methods focus on learning concepts from images. However, the visual information cannot present abstract concepts exactly, which struggles the introduction of novel concepts related to known concepts (e.g., 'Planet'→'Asteroids'). In this paper, inspired by the fact that humans learn most concepts through linguistic description, we introduce Linguistic Concept Learning benchmark (Licon), where concepts in diverse forms (e.g., plain attributes, images and text) are defined by linguistic descriptions. The difficulty to learn novel concepts can be controlled by the number of attributes or the hierarchical relationships between concepts. The diverse and controllable concepts are used to support challenging evaluation tasks, including concept classification, attribute prediction, and concept relationship recognition. In addition, we design an entailment-based concept learning method (EnC) to model the relationship among concepts. Extensive experiments demonstrate the effectiveness of EnC. The benchmark will be released to the public 1 .

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Cross-Task Generalization via Natural Language Crowdsourcing Instructions cites Multitask Prompted Training Enables Zero-Shot Task Generalization
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on10

  1. Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
  2. Multitask Prompted Training Enables Zero-Shot Task GeneralizationVictor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach et al.ICLR 2022 · 1,976 citations
  3. Cross-Task Generalization via Natural Language Crowdsourcing InstructionsSwaroop Mishra, Daniel Khashabi, Chitta Baral, Hannaneh HajishirziACL 2022 · 887 citations
  4. Zero-shot Text Classification via Reinforced Self-trainingZhiquan Ye, Yuxia Geng, Jiaoyan Chen, Jingmin Chen et al.ACL 2020 · 75 citations
  5. Learning from Explanations with Neural Execution TreeZiqi Wang, Yujia Qin, Wenxuan Zhou, Jun Yan et al.ICLR 2020 · 41 citations

Related papers

  1. CLUES: A Benchmark for Learning Classifiers using Natural Language ExplanationsRakesh R. Menon, Sayan Ghosh, Shashank SrivastavaACL 2022 · 13 citations
  2. ConCon-Chi: Concept-Context Chimera Benchmark for Personalized Vision-Language TasksAndrea Rosasco, Stefano Berti, Giulia Pasquale, Damiano Malafronte et al.CVPR 2024 · 1 citation
  3. Concept-pedia: a Wide-coverage Semantically-annotated Multimodal DatasetKarim Ghonim, Andrei Stefan Bejgu, Alberte Fernández-Castro, Roberto NavigliEMNLP 2025
  4. Attribute-formed Class-specific Concept Space: Endowing Language Bottleneck Model with Better Interpretability and ScalabilityJianyang Zhang, Qianli Luo, Guowu Yang, Wenjing Yang et al.CVPR 2025
  5. Concept-Based Label Embedding via Dynamic Routing for Hierarchical Text ClassificationXuepeng Wang, Li Zhao, Bing Liu, Tao Chen et al.ACL 2021
Dusk over the sea between two cliffs drawn in fine vertical lines