Augmenting Realistic Charts with Virtual Overlays

In this paper, we introduce the concept of realistic charts, referring to charts in the real world that cannot be digitally altered, such as those printed in newspapers or used in PDF documents. By enabling interaction with and virtual enhancement of these realistic charts as if they were digital, we transform realistic charts into "digital charts"by adding virtual overlays. To achieve this, we identify 34 overlay strategies (e.g., highlights and trendlines) for five widely-used chart types (e.g., line charts) through systematic exploration and a formative study. To simplify overlay creation, we introduce a new grammar named Vega-Overlay. Leveraging this design space and grammar, we develop a system called HARVis, which allows users to generate virtual overlays through augmented reality devices using speech and optional gestures. A user study involving 33 participants from diverse fields, across 17 tasks, demonstrates the effectiveness and usability of HARVis.