MSR Asia理论中心前沿系列讲座 | 直播:神经网络的局部弹性及其所启发的理论

2022-08-29 | 作者:微软亚洲研究院

微软亚洲研究院理论中心前沿系列讲座第三期,将于8月25 日(本周四)上午10:00-11:00与你如约相见。这一期,我们请到了哥伦比亚大学的研究员邓准,带来关于神经网络的局部弹性及其所启发理论的分享,届时请锁定B 站“微软中国视频中心”直播间!


欢迎对理论研究感兴趣的老师同学们参与讲座并加入社区(加入方式见后文),共同推动理论研究进步,加强跨学科研究合作,助力打破 AI 发展瓶颈,实现计算机技术实质性发展!

直播地址:B 站“微软中国视频中心”直播间

直播时间:每两周直播一次,时间为周四上午 10:00-11:00(有变动将另行说明)

Zhun is a postdoctoral researcher with Toniann Pitassi and Richard Zemel at Columbia University, and also part of Simons Collaboration on the Theory of Algorithmic Fairness. Previously, Zhun got his Ph.D. in Computer Science at Harvard University, advised by Cynthia Dwork. His research interests lie at the intersection of theoretical computer science, machine learning, and social science. His work aims to make data science more trustworthy, statistically rigorous, and aligned with societal values.

报告题目: Local Elasticity of Neural Networks and Its Inspired Theory
报告摘要: In this talk, I will briefly review local elasticity of neural networks proposed by He et al. Then, based on that, I will introduce a new type of stability notion, which can improve over classical stability notions with respect to generalization behavior in certain situations. Specifically, among different notions of stability, uniform stability is arguably the most popular one, which yields exponential generalization bounds. However, uniform stability only considers the worst-case loss change (or so-called sensitivity) by removing a single data point, which is distribution-independent and therefore undesirable. There are many cases that the worst-case sensitivity of the loss is much larger than the average sensitivity taken over the single data point that is removed, especially in some advanced models such as random feature models or neural networks. Many previous works try to mitigate the distribution independent issue by proposing weaker notions of stability, however, they either only yield polynomial bounds or the bounds derived do not vanish as sample size goes to infinity. Given that, we propose locally elastic stability as a weaker and distribution-dependent stability notion, which still yields exponential generalization bounds. We further demonstrate that locally elastic stability implies tighter generalization bounds than those derived based on uniform stability in many situations by revisiting the examples of bounded support vector machines, regularized least square regressions, and stochastic gradient descent.