graphiti: Sketch-based Graph Analytics for Images and Videos

Graph analytics is currently performed using a combination of code, symbolic algebra, and network visualizations. The analyst has to work with symbolic and abstract forms of data to construct and analyze graphs. We locate unique design opportunities at the intersection of computer vision and graph analytics, by utilizing visual variables extracted from images/videos and some direct manipulation and pen interaction techniques. We also summarize commonly used graph operations and graphical representations (graphs, simplicial complexes, hypergraphs), and map them to a few brushes and direct manipulation actions. The mapping enables us to visually construct and analyze a wide range of graphs on top of images, videos, and sketches. The design framework is implemented as a sketch-based notebook interface to demonstrate the design possibilities. User studies with scientists from various fields reveal innovative use cases for such an embodied interaction paradigm for graph analytics.