Stiffness: A New Perspective on Generalization in Neural Networks

They measure how stiff a network is by looking at how a small gradient step on one example affects the loss on another. Stiffness is useful for diagnosing and characterizing generalization.

For more details, visit the source.