AAAI2024

Visual Language - Let the Product Say What You Want

Jiaying Wang, Shuailing Hao, Jing Shan, Xiaoxu Song

摘要

Visual language is a new language composed of tightly integrated textual and visual elements. Its emergence is a result of such driving forces as globalization, increasing complexity both in commerce and technology, and the convergence of vocabularies from many previously distinct fields. A brief history of innovations 23 Invention of the various components and conventions of visual language has taken place over centuries. Communication units, morphology, and syntax 51 Examination of the building blocks of visual language. A. Size: Communication units in visual language 55 Different sizes of visual language components-from icon to mural-suggest categories for subsequent linguistic analysis. B. Components: Morphology of visual language 65 Four approaches to identifying primitive elements of language are examined; a synthesis based on the concept of a visual unity is proposed. C. Combinations: Syntax of visual language 73 Preliminary investigation into visual language syntax reveals that it is informed and constrained by spatial dimensions and perceptual principles. Emerging semantics 93 How meaning arises in visual language. A. Semantics of tight integration 95 The major semantic theme of the book: how do visual elements and text work together to produce meaning? B. Semantics of visual metaphors 113 Correlation is found between visual metaphor and contemporary theory about verbal metaphor. C. Diagram semantics 123 The varieties of diagramming make fundamental contributions to the vocabulary and semantics of visual language. D. Semantics of cartooning 135 Cartooning has a wide variety of semantic conventions that are spreading rapidly around the world and that provide a rich vocabulary. E. Semantics of space, line, and composition 143 Arrangement of elements on a page or in 3 dimensions is important in discussions of how visual language creates meaning. F. Semantics of time 151 How visual language shows phenomena of change, transition, and time is informed by conventions from the world of film and illustration.