Chasing Meaning and/or Insight? A Survey on Evaluation Practices at the Intersection of Visualization and the Humanities

The intersection of visualization and the humanities (VISH) is marked by a tension between chasing analytical "insight" and interpretive "meaning." The effectiveness of visualization techniques hinges on established evaluation frameworks that assess both analytical utility and communicative efficacy, creating a potential mismatch with the non-positivist, interpretive aims of humanities scholarship. To examine how this tension manifests in practice, we systematically surveyed 171 VISH design studies to analyze their evaluation workflows and rigor according to standard practice. Our findings reveal recurring flaws, such as an overreliance on monomethod approaches, and show that higher-quality evaluations emerge from workflows that effectively triangulate diverse evidence. From these findings, we derive recommendations to refine quality and validation criteria for humanities visualizations, and juxtapose them to ongoing critical debates in the field, ultimately arguing for a paradigm shift that can reconcile the advantages of established validation techniques with the interpretive depth required for humanistic inquiry. CCS Concepts: • Human-centered computing → Visualization design and evaluation methods; Visualization theory, concepts and paradigms; Empirical studies in visualization; HCI design and evaluation methods; • General and reference → Surveys and overviews; • Applied computing → Arts and humanities.