About
- I have been working with Prof. Shengchao Liu since June 2025, and formally serve as a Research Assistant at Wave Intelligence Lab, the Department of Computer Science and Engineering, the Chinese University of Hong Kong, starting from Spring 2026.
- I was a visiting student at Hefei Institutes of Physical Science, Chinese Academy of Sciences, hosted by Prof. Jianhui Zhou, from July 2025 to Oct. 2025.
- I received my Bachelor of Science degree from the School of Physics, Sun Yat-sen University, in June 2025. During this period, I was a member of the PMI Lab and was advised by Prof. Haiping Huang from Oct. 2022 to June 2025.
Interests
- Theoretical Mechanism of LLM and Deep Learning
- Statistical Mechanics of Disordered Systems
- Mean Field Theory of Spin Glass Models (Replica Method, Cavity Method, etc.)
- Random Matrix Theory, Dynamical Mean Field Theory, Nonlinear Dynamics
- Applications of LLM in Science and Engineering
Publications
A Minimal Model of Representation Collapse: Frustration, Stop-Gradient, and Dynamics
Louie Hong Yao*, Yuhao Li*, Shengchao Liu
arXiv:2604.09979, 2026.Spin Glass Model of In-Context Learning
Yuhao Li, Ruoran Bai, and Haiping Huang
Phys. Rev. E 112, L013301, 2025.
Projects
A Minimal Model of Representation Collapse
We build a minimal dynamical model directly in representation space, abstracting away the details of network architecture and parameters. We use the concept of frustration from statistical physics to describe the core mechanism behind representation collapse, and analyze how Stop-Gradient can break the symmetry and open up a non-collapsing subspace that preserves geometric separation between classes.
View Project
Spin Glass Model of In-Context Learning
We mapped in-context learning in a linear attention model to a spin glass with real-valued spins, and solved the ground state, energy landscape and phase behavior to show how task diversity drives a unique solution that enables in-context prediction in pre-trained transformers.
View Project
Writing
Why Do Representations Collapse? Frustration and Dynamics from a Minimal Model
This post introduces our recent work. We build a minimal dynamical model directly in representation space, abstracting away the details of network architecture and parameters. We use the concept of frustration from statistical physics to describe the core mechanism behind representation collapse, and analyze how Stop-Gradient can break the symmetry and open up a non-collapsing subspace that preserves geometric separation between classes.
Read Post
Timeline of Deep Learning
A timeline highlighting key milestones in the development of deep learning.
Read Post
Phase Transition Point in Classical Dynamics
A brief derivation of the stability/chaos transition in a classical random dynamical system using mean-field theory.
Read Post
Timeline for Statistical Mechanics of Neural Networks
A timeline highlighting key developments in the application of statistical mechanics to neural networks.
Read Post
Replica Method for Boolean Perceptron with Continuous Weight
An brief derivation for the asymptotic generalization error of a Boolean perceptron with continuous weights using the replica method. This problem was also the final assignment in the 'Statistical Mechanics of Neural Networks' course at SYSU, Fall 2024.
Read Post
Replica Method for the Sherrington-Kirkpatrick Model
This note reviews the replica symmetric solution of the SK model, including sufficiently detailed derivations, and then shows the phase diagrams of the order parameters and free energy by numerical calculations.
Read Post