Shapir: Standardizing and Democratizing Access to Web APIs

Today, many web sites ofer third-party access to their data through web APIs. But manually encoding URLs with arbitrary endpoints, parameters, authentication handshakes, and pagination, among other things, makes API use challenging and laborious for programmers, and untenable for novices. In addition, each site ofers its own idiosyncratic data model, properties, and methods that a new user must learn, even when the sites manage the same common types of information as many others.

In this work, we show how working with web APIs can be dramatically simplifed by describing the APIs using a standardized, machine-readable ontology. By surveying a statistical sample of web APIs, we develop a simple ontology that can efectively describe the core functionality of nearly all of them. We then present Shapir, a system that includes a graphical, form-based authoring tool for the API description, from which Shapir can automatically generate a standardized JavaScript library for accessing data on the web site as objects with readable and writeable properties. This enables programmers to access data without learning the details of each API, and indeed allows them to use the same unchanged code for multiple web sites. We then integrate Shapir with Mavo, an HTML language extension for describing web applications declaratively, to also empower plain-HTML authors to access these APIs. In our lab evaluation, we found that programmers are able to accomplish program data management tasks that require multiple API requests 5.6 times faster on average using the Shapir generic library than using the popular Swagger API integration library. Using our Mavo-Shapir integration, even non-programmers were able to build functioning data management applications that access multiple web APIs in just 4 minutes.