hS: Speculative Script Reordering at Subprocess Granularity

Shell scripts are pervasive, acting as the glue between commands and subprocesses that are written in a variety of languages and perform complex, system-wide effects. Given the black-box nature of these subprocesses, all work that optimizes script performance until now has relied on handwritten annotations that describe subprocess effects. In this paper we introduce hS, a system that brings out-of-order, speculative execution to scripts that invoke subprocesses without requiring any user input or annotations about them. hS speculatively executes command instances-typically simple commands, pipelines, or small synchronization regions-dynamically detecting their effects: blocking unsafe ones, like network accesses, and selectively committing independent effects, like file writes, given no conflicts. On a wide range of real-world scripts, hS offers up to 9.3× speedups compared to bash, and up to 7× speedups over PaSh-all while not requiring any developer involvement or command annotations.

† Now affiliated with Columbia University.