EMNLP Findings2020
NLP Service APIs and Models for Efficient Registration of New Clients
Sahil Shah, Vihari Piratla, Soumen Chakrabarti, Sunita Sarawagi
Abstract
State-of-the-art NLP inference uses enormous neural architectures and models trained for GPU-months, well beyond the reach of most consumers of NLP. This has led to one-sizefits-all public API-based NLP service models by major AI companies, serving large numbers of clients. Neither (hardware deficient) clients nor (heavily subscribed) servers can afford traditional fine tuning. Many clients own little or no labeled data. We initiate a study of adaptation of centralized NLP services to clients, and present one practical and lightweight approach. Each client uses an unsupervised, corpus-based sketch to register to the service. The server uses an auxiliary network to map the sketch to an abstract vector representation, which then informs the main labeling network. When a new client registers with its sketch, it gets immediate accuracy benefits. We demonstrate the success of the proposed architecture using sentiment labeling, NER, and predictive language modeling.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on3
- Efficient Domain Generalization via Common-Specific Low-Rank DecompositionVihari Piratla, Praneeth Netrapalli, Sunita SarawagiICML 2020 · 250 citations
- MetaNER: Named Entity Recognition with Meta-LearningJing Li, Shuo Shang, Ling ShaoWWW 2020 · 56 citations
- Mogrifier LSTMGábor Melis, Tomás Kociský, Phil BlunsomICLR 2020
Related papers
- Transductive Learning for Textual Few-Shot Classification in API-based Embedding ModelsPierre Colombo, Victor Pellegrain, Malik Boudiaf, Myriam Tami et al.EMNLP 2023 · 7 citations
- Cache me if you Can: an Online Cost-aware Teacher-Student framework to Reduce the Calls to Large Language ModelsIlias Stogiannidis, Stavros Vassos, Prodromos Malakasiotis, Ion AndroutsopoulosEMNLP Findings 2023 · 5 citations
- Practical and Efficient Model Extraction of Sentiment Analysis APIsWeibin Wu, Jianping Zhang, Victor Junqiu Wei, Xixian Chen et al.ICSE 2023 · 10 citations
- PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPUYixin Song, Zeyu Mi, Haotong Xie, Haibo ChenSOSP 2024 · 86 citations
- Sketch to Adapt: Fine-Tunable Sketches for Efficient LLM AdaptationTianyi Zhang, Junda Su, Aditya Desai, Oscar Wu et al.ICML 2025
