ChartDialogs: Plotting from Natural Language Instructions

This paper presents the problem of conversational plotting agents that carry out plotting actions from natural language instructions. To facilitate the development of such agents, we introduce CHARTDIALOGS, a new multi-turn dialog dataset, covering a popular plotting library, matplotlib. The dataset contains over 15, 000 dialog turns from 3, 200 dialogs covering the majority of matplotlib plot types. Extensive experiments show the bestperforming method achieving 61% plotting accuracy, demonstrating that the dataset presents a non-trivial challenge for future research on this task.