Statistical Physics of Neural Networks

Transported from https://shihsherwang.quarto.pub/assignments/

Classic

• Storing Infinite Numbers of Patterns in a Spin-Glass Model of Neural Networks
  Daniel J. Amit, Hanoch Gutfreund, H. Sompolinsky
  Phys. Rev. Lett. 55, 1530 (1985)
• Statistical Mechanics of Neural Networks Near Saturation
  Daniel J. Amit, Hanoch Gutfreund, H. Sompolinsky
  Annals of Physics 173, 30 (1987)
• Chaos in Random Neural Networks
  H. Sompolinsky, A. Crisanti, H. J. Sommers
  Phys. Rev. Lett. 61, 259 (1988)
• Path Integral Approach to Random Neural Networks
  A. Crisanti, H. Sompolinsky
  Phys. Rev. E 98, 062120 (2018)
• Spectrum of Large Random Asymmetric Matrices
  H. J. Sommers, A. Crisanti, H. Sompolinsky, Y. Stein
  Phys. Rev. Lett. 60, 1895 (1988)
• The Space of Interactions in Neural Network Models
  E. Gardner
  J. Phys. A: Math. Gen. 21, 257 (1988)
• Optimal Storage Properties of Neural Network Models
  E. Gardner, B. Derrida
  J. Phys. A: Math. Gen. 21, 271 (1988)
• Learning from Examples in Large Neural Networks
  H. Sompolinsky, N. Tishby, H. S. Seung
  Phys. Rev. Lett. 65, 1683 (1990)
• Statistical Mechanics of Learning from Examples
  H. S. Seung, H. Sompolinsky, N. Tishby
  Phys. Rev. A 45, 6056 (1992)
• First-Order Transition to Perfect Generalization in a Neural Network with Binary Synapses
  Geza Gyorgyi
  Phys. Rev. A 41, 7097 (1990)
• Exact Solution for on-Line Learning in Multilayer Neural Networks
  David Saad, Sara A. Solla
  Phys. Rev. Lett. 74, 4337 (1995)
• Chaos in Neuronal Networks with Balanced Excitatory and Inhibitory Activity
  C. van Vreeswijk, H. Sompolinsky
  Science 274, 1724 (1996)
• Chaotic Balanced State in a Model of Cortical Circuits
  C. van Vreeswijk, H. Sompolinsky
  Neural Computation 10, 1321 (1998)
• Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons
  Nicolas Brunel
  J. Comput. Neurosci. 8, 183 (2000)
• Eigenvalue Spectra of Random Matrices for Neural Networks
  Kanaka Rajan, L. F. Abbott
  Phys. Rev. Lett. 97, 188104 (2006)
• A Statistical Mechanics Framework for Bayesian Deep Neural Networks Beyond the Infinite-Width Limit
  R. Pacelli, S. Ariosto, M. Pastore, F. Ginelli, M. Gherardi, P. Rotondo
  Nature Machine Intelligence 5, 1497 (2023)

Selected

• Stimulus-Dependent Suppression of Chaos in Recurrent Neural Networks
  Kanaka Rajan, L. F. Abbott, Haim Sompolinsky
  Phys. Rev. E 82, 011903 (2010)
• Dynamics of Random Neural Networks with Bistable Units
  M. Stern, H. Sompolinsky, L. F. Abbott
  Phys. Rev. E 90, 062710 (2014)
• Mean-Field Message-Passing Equations in the Hopfield Model and Its Generalizations
  Marc Mezard
  Phys. Rev. E 95, 022117 (2017)
• Linking Connectivity, Dynamics, and Computations in Low-Rank Recurrent Neural Networks
  F. Mastrogiuseppe, S. Ostojic
  Neuron 99, 609 (2018)
• High-Dimensional Dynamics of Generalization Error in Neural Networks
  Madhu S. Advani, Andrew M. Saxe, Haim Sompolinsky
  Neural Networks 132, 428 (2020)
• Unified Field Theoretical Approach to Deep and Recurrent Neuronal Networks
  Kai Segadlo, Bastian Epping, Alexander van Meegen, David Dahmen, Michael Kramer, Moritz Helias
  J. Stat. Mech. 103401 (2022)
• Dimension of Activity in Random Neural Networks
  David G. Clark, L. F. Abbott, Ashok Litwin-Kumar
  Phys. Rev. Lett. 131, 118401 (2023)
• Theory of Coupled Neuronal-Synaptic Dynamics
  David G. Clark, L. F. Abbott
  Phys. Rev. X 14, 021001 (2024)
• Introduction to dynamical mean-field theory of randomly connected neural networks with bidirectionally correlated couplings
  Wenxuan Zou, Haiping Huang
  SciPost Phys. Lect. Notes 79 (2024)
• Coding Schemes in Neural Networks Learning Classification Tasks
  Alexander van Meegen, Haim Sompolinsky
  Nature Communications 16, 3354 (2025)
• Transfer Learning in Infinite Width Feature Learning Networks
  Clarissa Lauditi, Blake Bordelon, Cengiz Pehlevan
  arXiv (2025)
• Fixed Point Geometry in Chaotic Neural Networks
  Jakob Stubenrauch, Christian Keup, Anno C. Kurth, Moritz Helias, Alexander van Meegen
  Phys. Rev. Res. 7, 023203 (2025)
• Dynamical Learning in Deep Asymmetric Recurrent Neural Networks
  Davide Badalotti, Carlo Baldassi, Marc Mezard, Mattia Scardecchia, Riccardo Zecchina
  arXiv (2025)
• Training Dynamics of in-Context Learning in Linear Attention
  Yedi Zhang, Aaditya K Singh, Peter E. Latham, Andrew M. Saxe
  ICML (2025)
• Learning Linear Regression with Low-Rank Tasks in-Context
  Kaito Takanami, Takashi Takahashi, Yoshiyuki Kabashima
  arXiv (2025)