Prof. Dr. Zohar Ringel

A unified field theory approach to feature learning and generalization

One of the main merits of field theory is its role as a common language for reasoning about physical systems. In this talk, I'll portray how it may play a similar role in deep learning. In the first part, we'll set up a general field theory formulation of Bayesian Neural Networks or Langevin-trained DNNs at equilibrium. The aim would be to reproduce various known results within this unifying perspective using standard field theory methods. We'll start by deriving the DNN to Gaussian-Process correspondence at infinite width and obtain the dataset-averaged Gaussian Process action. We would then discuss the actions associated with finite-width DNNs and how two types of mean-field approximations on the interaction/non-linear terms in those actions can explain some of the mysteries of deep learning: DNNs' ability to generalize well despite having infinite expressibility and DNNs' ability to learn functions with better sample complexity scaling than their corresponding infinite-width/GP limit.