Projects

Mapped in-context learning in a linear attention model to a spin glass with real-valued spins; solved the ground state, energy landscape and phase behavior to show how task diversity drives a unique solution that enables in-context prediction in pre-trained transformers.