With Visual Integrity and Care: A Framework for Mixed Methods Research on Visual Social Data

The internet is becoming increasingly visual, but social computing research and methodological training has relied heavily on textual methods. Methodological innovation is needed to study visual social data, including problematic information (mis-and disinformation, propaganda, hate, AI slop, etc). Contending with this, we present a framework for conducting grounded, interpretive, computationally supported, mixed-method research on collections of visual social media data. We developed this framework while grappling with the ethical, logistical, and methodological challenges of conducting in-depth analysis of potentially harmful visual content while caring for our research team. We document our framework components of visual grammars, human analysis, and computationally supported analysis with an umbrella commitment to care and its use in three empirical case studies. We also provide recommendations and implications for the HCI community in embracing training in and the advancing of visual methods and research, including a sensitizing concept of visual integrity.

• Human-centered computing → HCI design and evaluation methods; Collaborative and social computing design and evaluation methods.