Schedule-Free Learning - A New Way to Train Schedule-Free Optimizers in PyTorch. Authors: Aaron Defazio, Xingyu Yang, Konstantin Mishchenko, Ashok Cutkosky, Harsh Mehta, Ahmed Khaled TLDR Faster training without schedules - no need to specify the stopping time/steps in advance! pip install schedulefree Primary implementations are SGDScheduleFree and AdamWScheduleFree. Approach Schedule-Free learning replaces the momentum of an underlying optimizer with a combination of interpolation and averagin...