基本信息

要点解读

这是什么

  • 该研究指出，对用于图像重建的预训练视觉编码器进行微调会压缩其表示的有效维度，进而影响扩散模型在高层特征空间中的生成效率。
  • 为解决这一问题，文中建议在高层空间中放弃常规的速度预测，转而采用直接预测原始数据的 x₀ 参数化，使学习集中在信号流形上。

用在哪里

  • 适用于在文本转图像生成任务中结合预训练视觉编码器与扩散模型的研究团队。
  • 对关注表征维度变化对生成质量和训练效率影响的开发者也有参考价值。

可以推断的

  • 推测：在高层特征空间里，优化目标的选择对收敛速度和最终效果有显著影响。
  • 推测：x₀ 参数化在类似的表示压缩场景中可能带来更稳定的训练过程。

来源摘要/节选

Representation Autoencoders (RAEs) enable diffusion models to operate in the feature spaces of pretrained visual encoders. However, many off-the-shelf encoders are not optimized for faithful reconstruction, discarding fine-grained visual details. As expected, finetuning these encoders for image reconstruction recovers such details. However, perhaps counterintuitively, this procedure reduces the effective dimensionality of the resulting representation, and the altered geometry has downstream effects on generation. Specifically, we show that using the standard velocity prediction in flow matching in this high-dimensional space requires the model to fit orthogonal noise directions outside the low-dimensional signal manifold, making optimization inefficient. This motivates using the clean data parameterization ($\boldsymbol{x}{0}$-prediction) instead, which focuses learning on the underlying signal manifold. Across experiments with multiple strong-reconstruction encoders, we show that $\boldsymbol{x}{0}$-prediction consistently improves text-to-image generation performance.

来源说明

当前保存的是来源摘要，不代表论文全文。请以原始来源为准。

「要点解读」由 AI Stack 依据上方已保存内容整理，不代表来源的完整表述；标注「推测：」的判断来自编辑，不是来源陈述。