PROFIT: A Specialized Optimizer for Deep Fine Tuning

The fine-tuning of pre-trained models has become ubiquitous in generative AI, computer vision, and robotics. Although much attention has been paid to improving the efficiency of fine-tuning models, there has been less scholarship around fine-tuning specifically for improved model performance. To remedy this gap, we present PROFIT, one of the first optimizers designed to incrementally fine-tune converged models on new tasks and/or datasets. Unlike traditional optimizers such as SGD or Adam, which make minimal assumptions due to random initializations, PROFIT takes the properties of a converged model into account explicitly to regularize the optimization process. Employing a temporal gradient-orthogonalization process, PROFIT outperforms fine-tuning methods in various tasks, from image classification to multimodal language model training to large-scale motion prediction. Moreover, PROFIT is encapsulated as a modular optimizer, which makes it easy to integrate directly into any training pipeline with minimal engineering effort. maintaining performance on old tasks. We will also later show how to overcome this constraint even in non-proximal settings by introducing a warmup phase.

PROFIT (PROximal FIne Tuning) is shown schematically in Fig. 1. To the best of our knowledge, PROFIT is among the first optimizers explicitly designed for fine-tuning.

Our main contributions are as follows.

• We introduce PROFIT, an optimizer for fine-tuning converged models, easily integrated into any deep learning framework.

• We show that PROFIT allows unsupervised training as if the original data were available.

• We show that PROFIT outperforms standard fine-tuning methods on various tasks, from image classification to VLM fine-tuning to large-scale motion prediction for autonomous driving.