基本信息
- 来源: arxiv
- 原始来源: http://arxiv.org/abs/2608.02585v1
- 发布域名: arxiv.org
- 分类: cs.LG
- 作者: Zhaoxin Yu、Qi Shen、Hengli Li 等
要点解读
这是什么
GradCuit 在 Transformer 的选定层插入可优化的潜在状态，使奖励梯度能够直接回传到这些状态，从而在保持模型参数不变的情况下对内部推理进行细化。
用在哪里
该方法适用于需要在测试阶段提升大语言模型推理准确性和稳健性的研发场景，尤其适合关注内部推理过程可解释性或希望快速适配不同推理策略的开发者。
可以推断的
推测：通过直接优化潜在表示而非生成多个候选再挑选，可能在推理时更具计算效率。
推测：若能够在推理过程中聚焦于少数关键连接词，模型在少样本或跨领域迁移时可能表现更稳。
来源摘要/节选
Optimization-based latent reasoning improves large language model outputs by optimizing instance-specific continuous states at test time while keeping model parameters frozen. Existing methods, however, typically connect these states to the reasoning trajectory through decoded tokens, making sequence-level credit assignment indirect and obscuring how latent updates shape subsequent reasoning. We introduce GradCuit (gradient through circuit), which inserts optimizable latent states at a selected Transformer layer between the hidden representations of the prompt and the generated continuation. Causal self-attention provides every continuation-token log-probability with a differentiable path to every preceding latent state through the remaining Transformer blocks, enabling reward-weighted gradients from the entire continuation to be assigned directly to the latents. Across five instruction-tuned backbones, three reasoning benchmarks, and two answer formats, GradCuit achieves an average accuracy of 64.5%, outperforming chain-of-thought prompting by 6.6 percentage points and the strongest competing method by 2.4 points. GradCuit also demonstrates greater robustness: across seven learning-rate settings, it consistently outperforms LatentSeek while reducing the standard deviation of accuracy from 1.53 to 0.82, and even its random-walk variant remains competitive with LatentSeek. For interpretability, token-level gradient attribution reveals that latent influence concentrates on reasoning-connector tokens, while layer analysis identifies early-to-middle Transformer layers as the most effective optimization space. By directly optimizing internal reasoning from outcome feedback, GradCuit opens a new axis of robust and interpretable test-time scaling, where LLMs adapt how they reason rather than merely regenerate, sample, or rerank outputs.
来源说明
当前保存的是来源摘要，不代表论文全文。请以原始来源为准。
「要点解读」由 AI Stack 依据上方已保存内容整理，不代表来源的完整表述；标注「推测：」的判断来自编辑，不是来源陈述。