微软亚洲研究院理论中心前沿系列讲座第三期，将于8月25 日（本周四）上午10:00-11:00与你如约相见。这一期，我们请到了哥伦比亚大学的研究员邓准，带来关于神经网络的局部弹性及其所启发理论的分享，届时请锁定B 站“微软中国视频中心”直播间！

微软亚洲研究院理论中心前沿系列讲座是每两周进行一次的常设系列直播讲座，将邀请全球站在理论研究前沿的研究者介绍他们的研究发现，主题涵盖大数据、人工智能以及其他相关领域的理论进展。通过这一系列讲座，我们期待与各位一起探索当前理论研究的前沿发现，并建立一个活跃的理论研究社区。

欢迎对理论研究感兴趣的老师同学们参与讲座并加入社区（加入方式见后文），共同推动理论研究进步，加强跨学科研究合作，助力打破 AI 发展瓶颈，实现计算机技术实质性发展！

直播地址：B 站“微软中国视频中心”直播间

https://live.bilibili.com/730

直播时间：每两周直播一次，时间为周四上午 10:00-11:00（有变动将另行说明）

Zhun is a postdoctoral researcher with Toniann Pitassi and Richard Zemel at Columbia University, and also part of Simons Collaboration on the Theory of Algorithmic Fairness. Previously, Zhun got his Ph.D. in Computer Science at Harvard University, advised by Cynthia Dwork. His research interests lie at the intersection of theoretical computer science, machine learning, and social science. His work aims to make data science more trustworthy, statistically rigorous, and aligned with societal values.

报告题目: Local Elasticity of Neural Networks and Its Inspired Theory
报告摘要: In this talk, I will briefly review local elasticity of neural networks proposed by He et al. Then, based on that, I will introduce a new type of stability notion, which can improve over classical stability notions with respect to generalization behavior in certain situations. Specifically, among different notions of stability, uniform stability is arguably the most popular one, which yields exponential generalization bounds. However, uniform stability only considers the worst-case loss change (or so-called sensitivity) by removing a single data point, which is distribution-independent and therefore undesirable. There are many cases that the worst-case sensitivity of the loss is much larger than the average sensitivity taken over the single data point that is removed, especially in some advanced models such as random feature models or neural networks. Many previous works try to mitigate the distribution independent issue by proposing weaker notions of stability, however, they either only yield polynomial bounds or the bounds derived do not vanish as sample size goes to infinity. Given that, we propose locally elastic stability as a weaker and distribution-dependent stability notion, which still yields exponential generalization bounds. We further demonstrate that locally elastic stability implies tighter generalization bounds than those derived based on uniform stability in many situations by revisiting the examples of bounded support vector machines, regularized least square regressions, and stochastic gradient descent.

在上次讲座中，来自斯坦福大学的马腾宇教授介绍了他在预训练的理论理解方面的工作。特别的，他指出对比学习可以视为是在一个所谓总体正对图上进行谱聚类算法。来自微软的研究者们与外部观众积极踊跃的提出了自己对预训练的看法与疑问，并得到了马教授的解答。

回放地址：

https://www.bilibili.com/video/BV1TF411P7Wk?spm_id_from=333.999.0.0