The Surprising Victory of NLP: From Philosophy to Agentic Language Models

Language Models have been around for decades but have suddenly taken the world by storm. In a surprising third act for anyone doing Natural Language Processing (NLP) in the 70s, 80s, 90s, or 2000s, in much of the popular media, artificial intelligence is now synonymous with language models. In this talk, I want to take a look backward at where language models came from and why they were so slow to emerge, a look inward to give a few thoughts on meaning, intelligence, and what language models understand and know, and a look forward at some topics of recent research: the possibilities for progress with new versions of Universal Transformers and using Large Language Models (LLMs) to build intelligent language-using agents for the digital world. I will argue that material beyond language is not necessary to having meaning and understanding, but it is very useful in most cases, and that composability, adaptability, and learning are vital to intelligence. Rather than simply continuing to build ever-larger LLMs from huge passive collections of text data, we need to explore developing better neural architectures and agents that can learn through interactions. I will discuss how web agents can learn through interactions and how such an interaction-first learning approach can work very effectively in web environments with relatively small language models.