基本信息
- 来源: arxiv
- 原始来源: http://arxiv.org/abs/2608.23554v1
- 发布域名: arxiv.org
- 分类: cs.LG
- 作者: Daniil Dmitriev、Zhihan Huang、Yuting Wei
要点解读
这是什么
本文研究离散扩散模型中的采样方法，针对均匀和再掩码前向过程提出一种基于留一去噪器的一阶采样器，并证明其采样复杂度由目标分布的内部依赖结构（双总相关）决定，而不是直接取决于环境维度。
用在哪里
适合从事离散数据（如文本、代码、分子结构等）生成模型研发的工程师，以及关注在高维离散空间中提升采样效率的理论研究者。
可以推断的
- 推测：当目标分布的依赖关系较弱（双总相关较小）时，采样所需的离散化步骤可能更少，从而加快生成速度。
- 推测：在维度较高的离散空间里，这类采样器相比传统 τ‑leaping 采样器更不易受维度线性增长的影响，可能更具可扩展性。
来源摘要/节选
Discrete diffusion models offer a promising alternative to autoregressive generation by enabling parallel updates, but their sampling efficiency can depend strongly on the choice of the forward process and the sampler. For the uniform forward process, existing lower bounds for the standard $τ$-leaping sampler scale linearly with the ambient dimension $d$, raising the question of whether this dependence is intrinsic to the forward process. We answer this question in the negative. We consider a first-order sampler based on the leave-one-out denoiser for uniform and remasking processes whose coordinate updates can be performed in parallel. In both cases, the sampler can correct denoising mistakes during the sampling process, which becomes necessary when many coordinates are updated together. Our main result establishes an adaptive sampling guarantee: up to logarithmic factors, $N = O(\mathrm{DTC}(X_0) / \varepsilon)$ discretization steps suffice to achieve sampling error $O(\varepsilon_{\mathrm{score}}+\varepsilon)$, where $\varepsilon_{\mathrm{score}}$ is the error in score estimation. Thus, the sampling complexity is governed by the intrinsic dependence structure of the target distribution, as measured by its dual total correlation $\mathrm{DTC}(X_0)$, rather than directly by the ambient dimension $d$. Our analysis proceeds through a Bayes-optimal auxiliary sampler that separates discretization error from score-estimation error. We also derive an exact information-theoretic representation of the discretization error in terms of the mutual information between different coordinates of the forward process at different times. This representation applies to general forward processes and, in the uniform and remasking cases, can be controlled by $\mathrm{DTC}(X_0)$. Numerical experiments on structured synthetic distributions illustrate the predicted dimension-adaptive behavior.
来源说明
当前保存的是来源摘要，不代表论文全文。请以原始来源为准。
「要点解读」由 AI Stack 依据上方已保存内容整理，不代表来源的完整表述；标注「推测：」的判断来自编辑，不是来源陈述。