The asymptotic spectrum of the Hessian of DNN throughout training

Add code
Oct 01, 2019
Arthur Jacot, Franck Gabriel, Clément Hongler
Figure 1 for The asymptotic spectrum of the Hessian of DNN throughout training
Figure 2 for The asymptotic spectrum of the Hessian of DNN throughout training
Figure 3 for The asymptotic spectrum of the Hessian of DNN throughout training

Share this with someone who'll enjoy it:

Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon

The dynamics of DNNs during gradient descent is described by the so-called Neural Tangent Kernel (NTK). In this article, we show that the NTK allows one to gain precise insight into the Hessian of the cost of DNNs: we obtain a full characterization of the asymptotics of the spectrum of the Hessian, at initialization and during training.

View paper onarxiv iconopen_review iconOpenReview

Share this with someone who'll enjoy it:

Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon