ICML 2025

Right Time to Learn: Promoting Generalization via Bio-inspired Spacing Effect in Knowledge Distillation

Guanglong Sun*, Hongwei Yan*, Liyuan Wang, Qian Li, Bo Lei, Yi Zhong

Introduces a time interval between teacher and student updates in online and self-distillation, connecting improved generalization to flatter optimization landscapes.

Citation

Guanglong Sun*, Hongwei Yan*, Liyuan Wang, Qian Li, Bo Lei, Yi Zhong. Right Time to Learn: Promoting Generalization via Bio-inspired Spacing Effect in Knowledge Distillation. ICML, 2025.