基本信息
- 来源: arxiv
- 原始来源: https://arxiv.org/abs/2603.03099v1
- 作者: Ruinan Jin, Yingbin Liang, Shaofeng Zou
- 分类: cs.LG
- 论文时间: 2026-03-03T15:34:51Z
- 论文 PDF: https://arxiv.org/pdf/2603.03099v1.pdf
来源摘要/节选
Despite Adam demonstrating faster empirical convergence than SGD in many applications, much of the existing theory yields guarantees essentially comparable to those of SGD, leaving the empirical performance gap insufficiently explained. In this paper, we uncover a key second-moment normalization in Adam and develop a stopping-time/martingale analysis that provably distinguishes Adam from SGD under the classical bounded variance model (a second moment assumption). In particular, we establish the first theoretical separation between the high-probability convergence behaviors of the two methods: Adam achieves a $δ^{-1/2}$ dependence on the confidence parameter $δ$, whereas corresponding high-probability guarantee for SGD necessarily incurs at least a $δ^{-1}$ dependence.
来源说明
当前只保存了官方论文摘要，不代表论文全文。请以原始来源为准。
本页只呈现已做哈希绑定的来源证据，不包含基于旧正文或缺失原文的扩展推断。