Fast Direct Manipulation Programming with Patch-Reconciliation Correspondence

Direct manipulation programming gives users a way to write programs without directly writing code, by using the familiar GUI-style interactions they know from direct manipulation interfaces. To date, direct manipulation programming systems have relied on two core components: (1) a patch component, which modifies the program based on a GUI interaction, and (2) a forward evaluator, which executes the modified program to produce an updated program output. This architecture has worked for developing short-running programs-i.e., programs that reliably execute in <1 second-generating outputs such as SVG and HTML documents. However, direct manipulation programming has not yet been applied to long-running programs (e.g., data visualization, mapping), perhaps because executing such programs in response to every GUI interaction would mean crossing outside of interactive speeds. We propose extending direct manipulation programming to long-running programs by pairing a standard patch component (patch) with a corresponding reconciliation component (recon). recon directly updates the program output in response to a GUI interaction, obviating the need for forward evaluation.

We introduce corresponding patch and recon procedures for the domain of geospatial data visualization and prove them sound-that is, we show that the output produced by recon is identical to the output produced by forward-evaluating a patch-modified program. recon can operate both incrementally and in parallel with patch. Our implementation of our patch-recon instantiation achieves a 2.92× median reduction in interface latency compared to forward evaluation on a suite of real-world geospatial visualization tasks. Looking forward, our results suggest that patch-reconciliation correspondence offers a promising pathway for extending direct manipulation programming to domains involving large-scale computation.

CCS Concepts: • Human-centered computing → User interface programming; • Software and its engineering → Graphical user interface languages; Integrated and visual development environments.