Yuhao Li

Yuhao Li

Research Assistant at CSE, CUHK

Email GitHub Google Scholar

About

  • I have been working with Prof. Shengchao Liu since June 2025, and formally serve as a Research Assistant at Wave Intelligence Lab, the Department of Computer Science and Engineering, the Chinese University of Hong Kong, starting from Spring 2026.
  • I was a visiting student at Hefei Institutes of Physical Science, Chinese Academy of Sciences, hosted by Prof. Jianhui Zhou, from July 2025 to Oct. 2025.
  • I received my Bachelor of Science degree from the School of Physics, Sun Yat-sen University, in June 2025. During this period, I was a member of the PMI Lab and was advised by Prof. Haiping Huang from Oct. 2022 to June 2025.

Interests

My research interests lie at the intersection of artificial intelligence and statistical physics. I focus on the theoretical mechanisms of LLMs and deep learning, using tools from complex systems, spin-glass theory, random matrix theory, dynamical mean-field theory, and nonlinear dynamics. I am also I am also developing a broad interest in AI for science.

Publications

  • On Distinguishing Capability Elicitation from Capability Creation in Post-Training: A Free-Energy Perspective
    Yuhao Li, Shengchao Liu
    arXiv:2605.08368, 2026.

  • A Minimal Model of Representation Collapse: Frustration, Stop-Gradient, and Dynamics
    Louie Hong Yao*, Yuhao Li*, Shengchao Liu
    arXiv:2604.09979, 2026.

  • Spin Glass Model of In-Context Learning
    Yuhao Li, Ruoran Bai, and Haiping Huang
    Phys. Rev. E 112, L013301, 2025.

Projects

A Minimal Model of Representation Collapse

We build a minimal dynamical model directly in representation space, abstracting away the details of network architecture and parameters. We use the concept of frustration from statistical physics to describe the core mechanism behind representation collapse, and analyze how Stop-Gradient can break the symmetry and open up a non-collapsing subspace that preserves geometric separation between classes.

Spin Glass Model of In-Context Learning

We mapped in-context learning in a linear attention model to a spin glass with real-valued spins, and solved the ground state, energy landscape and phase behavior to show how task diversity drives a unique solution that enables in-context prediction in pre-trained transformers.
View All Projects

Writing

Why Do Representations Collapse? Frustration and Dynamics from a Minimal Model

This post introduces our recent work. We build a minimal dynamical model directly in representation space, abstracting away the details of network architecture and parameters. We use the concept of frustration from statistical physics to describe the core mechanism behind representation collapse, and analyze how Stop-Gradient can break the symmetry and open up a non-collapsing subspace that preserves geometric separation between classes.

Timeline of Deep Learning

A timeline highlighting key milestones in the development of deep learning.

Phase Transition Point in Classical Dynamics

A brief derivation of the stability/chaos transition in a classical random dynamical system using mean-field theory.

Timeline for Statistical Mechanics of Neural Networks

A timeline highlighting key developments in the application of statistical mechanics to neural networks.

Replica Method for Boolean Perceptron with Continuous Weight

An brief derivation for the asymptotic generalization error of a Boolean perceptron with continuous weights using the replica method. This problem was also the final assignment in the 'Statistical Mechanics of Neural Networks' course at SYSU, Fall 2024.

Replica Method for the Sherrington-Kirkpatrick Model

This note reviews the replica symmetric solution of the SK model, including sufficiently detailed derivations, and then shows the phase diagrams of the order parameters and free energy by numerical calculations.
View All Posts