ZurichNLP #14

Jonas Hübotter from ETH Zurich on Efficiently Learning at Test-Time with LLMs: The standard paradigm of machine learning separates training and testing. Training aims to learn a model by inductively extracting general rules from data, and testing applies this model to new, unseen data. We investigate an alternative transductive paradigm where the model is fine-tuned at test-time specifically to the given task. Our evaluation on the Pile dataset indicates that this paradigm can improve language modeling on a wide range of tasks. We identify the key challenge of deciding which data to select for test-time fine-tuning and show that the previously used Nearest Neighbor retrieval is ill-suited since it tends to select redundant data. To address this, we introduce SIFT, a data selection algorithm designed to reduce uncertainty about the model's response given a prompt, which unifies ideas from retrieval and active learning. Whereas Nearest Neighbor retrieval typically fails in the…