"Advancements in In-Context Learning: Transition from Few-Shot to Many-Shot Learning Shows Significant Performance Gains, Overrides Training Biases"

Many-Shot In-Context Learning

Authors:Rishabh Agarwal, Avi Singh, Lei M. Zhang, Bernd Bohnet, Stephanie Chan, Ankesh Anand, Zaheer Abbas, Azade Nova, John D. Co-Reyes, Eric Chu, Feryal Behbahani, Aleksandra Faust, Hugo Larochelle View PDF HTML (experimental) Abstract:Large language models (LLMs) excel at few-shot in-context learning (ICL) -- learning from a few examples provided in context at inference, without any weight updates. Newly expanded context windows allow us to investigate ICL with hundreds or thousands of exampl...