About Me
Yaoyu Zhang 张耀宇
I am a tenure-track Associate Professor at the Institute of Natural Sciences and the School of Mathematical Sciences at Shanghai Jiao Tong University. I received my B.S. in Applied Physics, with a minor in Applied Mathematics, in 2012 and my Ph.D. in Applied Mathematics in 2016, both from Shanghai Jiao Tong University. My doctoral advisors were David Cai and Douglas Zhou. From 2016 to 2020, I conducted postdoctoral research at New York University Abu Dhabi, the Courant Institute of Mathematical Sciences at New York University, and the Institute for Advanced Study in Princeton. My research focuses on the theoretical foundations of deep learning, particularly nonlinear training dynamics and the condensation phenomenon.
Research
Research Interests
Deep Learning Theory
- Nonlinear training dynamics and implicit bias
- Condensation phenomenon and theory
- Loss landscape structure
- Optimization dynamics
My research studies the theoretical foundations of deep learning through nonlinear training dynamics. A central focus is the condensation phenomenon, in which neurons in the same layer become aligned during training. Our work examines the regimes and early stages of condensation, loss landscape structure through the Embedding Principle, and sample efficiency through the Optimistic Estimate. Recent projects also investigate optimization dynamics, including loss spikes in Adam and convergence on degenerate objectives. My goal is to develop a more unified understanding of how training dynamics shape the solutions learned by neural networks.
Publications
Recent Work
Adaptive Preconditioners Trigger Loss Spikes in Adam
Zhiwei Bai, Zhangchen Zhou, Jiajie Zhao, Xiaolong Li, Zhiyu Li, Feiyu Xiong, Hongkang Yang, Yaoyu Zhang, Zhi-Qin John Xu.
Towards Understanding Adam Convergence on Highly Degenerate Polynomials
Zhiwei Bai, Jiajie Zhao, Zhangchen Zhou, Zhi-Qin John Xu, Yaoyu Zhang.
Gradient Flow Dynamics and Implicit Bias of Diagonal Linear Networks under Infinitesimal Initialization
Jiajie Zhao, Jianxing Wang, Junjie Yang, Zhiwei Bai, Yaoyu Zhang.
Architecture Induces Structural Invariant Manifolds of Neural Network Training Dynamics
Jiajie Zhao, Tao Luo, Yaoyu Zhang.
Resources
Selected Materials
Overview of Current Research Program
Condensation
An overview of our research on condensation, from dynamical regimes and loss landscape structure to sample efficiency, implicit bias, and language models. These connections guide our ongoing work toward a more unified theory of neural network training. See Publications for details.
Short Courses
Towards a Mathematical Foundation of Deep Learning: From Phenomena to Theory. (web)
Tianyuan Mathematical Center in Northeast China, Jilin University, Changchun, China, Sep. 13-15, 2026.
深度学习的数学基础(从现象到理论) (web)
Graduate Summer School, Xiangtan University, Xiangtan, China, Jul. 13-17, 2026.
Precourse Lecture at MLPDES25:
The condensation phenomenon of Deep Neural Networks (slides, video)
Towards a Mathematical Foundation of Deep Learning: From Phenomena to Theory. (web)
FAU MoD Course, Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany, May 2-8, 2025.
Session titles:
Selected Presentations
- Towards the Foundation of Deep Learning. XAGI 2026, Physics of Deep Learning session, Beijing, China, Oct. 18, 2026. (web)
- The Condensation Phenomenon of Deep Neural Networks. MaD Seminar, New York University, U.S.A., Feb. 13, 2025. (slides)
- Optimistic Sample Size Estimate for Deep Neural Networks. One World Seminar Series on the Mathematics of Machine Learning, Oct. 9, 2024. (slides, video)
- Embedding Principle of Loss Landscape of Deep Neural Networks. Online, 机器学习联合研讨计划, Oct. 16, 2021. (slides, video)
- Dynamics of Deep Neural Networks--A Fourier Analysis Perspective. Institute for Advanced Study, Princeton, Oct. 4, 2019. (slides, video)
Textbook
深度学习现象导论:从感知机到大模型
Deep Learning Phenomena: From Perceptrons to Large Models
Zhi-Qin John Xu (许志钦) and Yaoyu Zhang (张耀宇). Publishing House of Electronics Industry (电子工业出版社), 2026. In Chinese.
English PDF · Chinese lecture notes (2025) · Git Repo · Douban
Artworks