Yaoyu Zhang bio photo

Email

My CV

Github

Google Scholar

About Me

Yaoyu Zhang 张耀宇

Portrait of Yaoyu Zhang

I am a tenure-track Associate Professor at the Institute of Natural Sciences and the School of Mathematical Sciences at Shanghai Jiao Tong University. I received my B.S. in Applied Physics, with a minor in Applied Mathematics, in 2012 and my Ph.D. in Applied Mathematics in 2016, both from Shanghai Jiao Tong University. My doctoral advisors were David Cai and Douglas Zhou. From 2016 to 2020, I conducted postdoctoral research at New York University Abu Dhabi, the Courant Institute of Mathematical Sciences at New York University, and the Institute for Advanced Study in Princeton. My research focuses on the theoretical foundations of deep learning, particularly nonlinear training dynamics and the condensation phenomenon.

Research

Research Interests

Deep Learning Theory

  • Nonlinear training dynamics and implicit bias
  • Condensation phenomenon and theory
  • Loss landscape structure
  • Optimization dynamics

My research studies the theoretical foundations of deep learning through nonlinear training dynamics. A central focus is the condensation phenomenon, in which neurons in the same layer become aligned during training. Our work examines the regimes and early stages of condensation, loss landscape structure through the Embedding Principle, and sample efficiency through the Optimistic Estimate. Recent projects also investigate optimization dynamics, including loss spikes in Adam and convergence on degenerate objectives. My goal is to develop a more unified understanding of how training dynamics shape the solutions learned by neural networks.

Resources

Selected Materials

Overview of Current Research Program

Condensation research program Five research tracks connect condensation to regimes, loss landscape, generalization, global dynamics, and language models. Condensation A1 Regime I Phase diagram two-layer II Three-layer A2 Loss landscape III Near 0 I, II Embedding principle IV Embedding in depth V, VI Critical sets VII KKT embedding A3 Generalization I, II Optimistic estimate III Local linear recovery IV Local recovery A4 Global dynamics I, II Implicit bias III One neuron recovery IV Structural invariant manifold A5 Language model I Initialization impact II, IV Reasoning III Compositional generalization

Condensation

A1 · RegimeI Phase diagram, two-layer → II Three-layer
A2 · Loss landscapeIII Near 0 · I, II Embedding principle (IV Embedding in depth · V, VI Critical sets · VII KKT embedding)
A3 · GeneralizationI, II Optimistic estimate → III Local linear recovery; IV Local recovery
A4 · Global dynamicsI, II Implicit bias · III One neuron recovery · IV Structural invariant manifold
A5 · Language modelI Initialization impact → II, IV Reasoning; III Compositional generalization

An overview of our research on condensation, from dynamical regimes and loss landscape structure to sample efficiency, implicit bias, and language models. These connections guide our ongoing work toward a more unified theory of neural network training. See Publications for details.

Short Courses

Towards a Mathematical Foundation of Deep Learning: From Phenomena to Theory. (web)

Tianyuan Mathematical Center in Northeast China, Jilin University, Changchun, China, Sep. 13-15, 2026.

深度学习的数学基础(从现象到理论) (web)

Graduate Summer School, Xiangtan University, Xiangtan, China, Jul. 13-17, 2026.

Precourse Lecture at MLPDES25:

The condensation phenomenon of Deep Neural Networks (slides, video)

Towards a Mathematical Foundation of Deep Learning: From Phenomena to Theory. (web)

FAU MoD Course, Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany, May 2-8, 2025.

Session titles:

  1. Mysteries of Deep Learning (slides, video)
  2. Frequency Principle/Spectral Bias (slides, video)
  3. Condensation Phenomenon (slides, video)
  4. From Condensation to Loss Landscape Analysis (slides, video)
  5. From Condensation to Generalization Theory (slides, video)

Selected Presentations

  • Towards the Foundation of Deep Learning. XAGI 2026, Physics of Deep Learning session, Beijing, China, Oct. 18, 2026. (web)
  • The Condensation Phenomenon of Deep Neural Networks. MaD Seminar, New York University, U.S.A., Feb. 13, 2025. (slides)
  • Optimistic Sample Size Estimate for Deep Neural Networks. One World Seminar Series on the Mathematics of Machine Learning, Oct. 9, 2024. (slides, video)
  • Embedding Principle of Loss Landscape of Deep Neural Networks. Online, 机器学习联合研讨计划, Oct. 16, 2021. (slides, video)
  • Dynamics of Deep Neural Networks--A Fourier Analysis Perspective. Institute for Advanced Study, Princeton, Oct. 4, 2019. (slides, video)

Textbook

深度学习现象导论:从感知机到大模型
Deep Learning Phenomena: From Perceptrons to Large Models

Zhi-Qin John Xu (许志钦) and Yaoyu Zhang (张耀宇). Publishing House of Electronics Industry (电子工业出版社), 2026. In Chinese.

English PDF · Chinese lecture notes (2025) · Git Repo · Douban

Artworks

Framed artwork with colorful handwritten ribbons and swirling patterns, gifted to me by my students.
Artwork Gifted to Me by My Students