Abstraction Alignment: Comparing Model-Learned and Human-Encoded Conceptual Relationships

Angie W. Boggust; Hyemin Bang; Hendrik Strobelt; Arvind Satyanarayan

doi:10.1145/3706598.3713406

CHI2025顶会

Abstraction Alignment: Comparing Model-Learned and Human-Encoded Conceptual Relationships

Angie W. Boggust, Hyemin Bang, Hendrik Strobelt, Arvind Satyanarayan

arXiv DOI 出版方

2025年份

4被引次数

2顶会引用

摘要

A model's confidence distribution is a reflection of its underlying knowledge.

Human abstractions represent the concepts and relationships we expect models to learn.

Abstraction alignment measures how much of a model's uncertainty can be explained by the human abstractions. SUBGRAPH PREFERENCE: Confidence in different regions of the abstraction. ABSTRACTION MATCH: Uncertainty reduced by a level of abstraction. CONCEPT CO-CONFUSION: Concepts the model regularly confuses.