基本信息
- 来源: arxiv
- 原始来源: https://arxiv.org/abs/2602.19345v1
- 作者: Egor Denisov, Svetlana Glazyrina, Maksim Kryzhanovskiy, Roman Ischenko
- 分类: cs.LG
- 论文时间: 2026-02-22T21:19:26Z
- 论文 PDF: https://arxiv.org/pdf/2602.19345v1.pdf
来源摘要/节选
Group Relative Policy Optimization (GRPO) has significantly advanced the training of large language models and enhanced their reasoning capabilities, while it remains susceptible to instability due to the use of hard clipping. Soft Adaptive Policy Optimization (SAPO) addresses this limitation by replacing clipping with a smooth sigmoid-based gate function, which leads to more stable updates. We have decided to push this theory further and investigate the impact of different gate functions on both training stability and final model performance. We formalize the key properties that admissible gates should satisfy and identify several families of such functions for empirical evaluation. This paper presents an analysis of our findings based on experiments conducted with the Qwen2.5-7B-Instruct model on mathematical reasoning tasks. These results provide practical guidance for designing smoother and more robust policy optimization objectives for large language model training.
来源说明
当前只保存了官方论文摘要，不代表论文全文。请以原始来源为准。
本页只呈现已做哈希绑定的来源证据，不包含基于旧正文或缺失原文的扩展推断。