基本信息
- 来源: arxiv
- 原始来源: http://arxiv.org/abs/2608.23551v1
- 发布域名: arxiv.org
- 分类: cs.CL
- 作者: Na Li、Yuchen Jiao、Changxiao Cai 等
要点解读
这是什么
ConvergeFlow 是一种在嵌入空间进行流式建模的语言模型，它把数据预测器限制在词向量凸包内，仅使用流匹配诱导的均方误差进行训练，实现流向有效词向量的收敛，从而可以在不依赖交叉熵监督解码器的情况下直接预测词。
用在哪里
该研究适用于从事文本生成和流模型研究的科研人员与工程师，尤其是想摆脱对交叉熵解码器依赖、探索连续扩散与流式语言模型实际应用的人群。
可以推断的
推测：在保持生成质量的前提下，使用均方误差训练可能简化模型的优化过程并提升训练稳定性。
推测：该方法的收敛保证如果在更大规模数据或更长序列上得到验证，可能为工业级流式文本生成系统提供可行的技术路径。
来源摘要/节选
Recent advances in continuous diffusion and flow-based language models (LMs) have achieved performance competitive with discrete LMs. However, existing continuous frameworks still rely on decoders supervised with cross entropy (CE) because the flow trajectories are not guaranteed to terminate at valid token embeddings. Motivated by this limitation, we introduce \textbf{ConvergeFlow}, an embedding-space flow-based LM, which constrains the data predictor to the convex hull of token embeddings and trains it solely with the mean squared error objective induced by flow matching. Under suitable regularity conditions, we prove that the resulting flow converges to valid token embeddings despite errors in the data predictor, enabling direct token prediction without a CE-supervised decoder. We further develop three sampling mechanisms for controlling the trade-off between the generative perplexity and entropy. Experiments on OpenWebText demonstrate that ConvergeFlow achieves performance competitive with existing continuous and discrete diffusion LMs. These findings demonstrate the potential of the flow-based paradigm for language modeling. Our code is available at https://github.com/Na-Li66/ConvergeFlow .
来源说明
当前保存的是来源摘要，不代表论文全文。请以原始来源为准。
「要点解读」由 AI Stack 依据上方已保存内容整理，不代表来源的完整表述；标注「推测：」的判断来自编辑，不是来源陈述。