基本信息

要点解读

这是什么

ArchSpace 是一个面向大语言模型架构研究的公开平台,用于记录研究提案、实验过程与推理,并通过社区评审和后续引用形成可追溯的积累。它还用缩放定律从较小规模实验预测更大规模的模型损失,以辅助早期评估。

用在哪里

该平台适合希望在投入大规模训练前验证架构想法的研究人员与模型开发团队,也可用于查找可复用的训练配方、基线和经过评审的研究记录。

可以推断的

  • 推测:这种公开记录与评审机制可能有助于减少不同研究之间的重复尝试,并提升架构实验的可复现性。
  • 推测:基于缩放定律的预测可作为筛选候选方案的参考,但最终效果仍可能需要更大规模训练验证。

来源摘要/节选

Model architecture shapes the scaling behavior, training efficiency, and inference cost of LLMs, making it a central focus of both industry and academia. In industry, teams such as those behind DeepSeek, Qwen, and Kimi routinely introduce architectural innovations, yet public reports rarely document which ideas succeeded, which were abandoned, and why. In academia, limited compute and inconsistent baselines often restrict evaluations to small-scale, single-point comparisons, making it difficult to establish whether improvements hold at larger scales or translate into broader impact. ArchSpace addresses both gaps as a public platform for Recording, Reviewing, and Referencing Architecture Researc}: each proposal is recorded with its experiments and reasoning, reviewed by the community, and referenced by later work. To evaluate ideas before costly large-scale training, ArchSpace leverages scaling laws to predict model loss at larger scales from smaller experiments, a practice widely adopted in industry. We provide initial references that make the platform usable: Qwen3.5 and OLMo3 baselines, fitted scaling laws, and a reusable training recipe. Ideas validated through this review can in turn become new baselines, and ArchSpace thereby aims to broaden the range of ideas that shape the next generation of models.

来源说明

当前保存的是来源摘要,不代表论文全文。请以原始来源为准。

「要点解读」由 AI Stack 依据上方已保存内容整理,不代表来源的完整表述;标注「推测:」的判断来自编辑,不是来源陈述。