An Analysis of the Costs and Benefits of Autocomplete in IDEs

Many IDEs support an autocomplete feature, which may increase developer productivity by reducing typing requirements and by providing convenient access to relevant information. However, to date, there has been no evaluation of the actual benefit of autocomplete to programmers. We conducted a between-subjects experiment <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> mml:mo(</mml:mo> mml:mtextN</mml:mtext> mml:mo=</mml:mo> mml:mn32</mml:mn> mml:mo)</mml:mo> </mml:math> using an eye tracker to evaluate the costs and benefits of IDE-based autocomplete features to programmers who use an unfamiliar API. Participants who used autocomplete spent significantly less time reading documentation and got significantly higher scores on our post-study API knowledge test, indicating that it helped them learn more about the API. However, autocomplete did not significantly reduce the number of keystrokes required to finish tasks. We conclude that the primary benefit of autocomplete is in providing information, not in reducing time spent typing.