NLP Libraries, Energy Consumption and Runtime: An Empirical Study

In the realm of natural language processing (NLP), the rising computational demands of modern models bring energy efficiency to the forefront of sustainable computing. Preprocessing tasks, such as tokenization, stemming, and POS tagging, are critical steps in transforming raw text into structured formats suitable for machine learning models. However, despite their widespread use in numerous NLP pipelines, little attention has been given to their energy consumption. This empirical study evaluates and compares the energy consumption and runtime performance of three popular NLP libraries— NLTK , spaCy , and Gensim —across six common preprocessing tasks. We conducted a comprehensive comparison using three distinct datasets and six preprocessing tasks. Energy consumption was measured using the Intel-RAPL and NVIDIA-SMI interfaces, while runtime performance was recorded across all library-task combinations. The results reveal substantial discrepancies in energy consumption across the three libraries, with up to 93% of cases exhibiting significant variations. Gensim showed superior efficiency in tokenization and stemming, while spaCy excelled in tasks like POS tagging and Named Entity Recognition (NER). These findings underscore the potential for optimizing NLP preprocessing tasks for energy efficiency. Our study highlights the untapped potential for improving energy efficiency in NLP pipelines. These insights emphasize the need for more focused research into energy-efficient NLP techniques, especially in the preprocessing phase, to support the development of greener, more sustainable computational models.