基本信息

要点解读

这是什么

该工作提出一种在移动端为大型语言模型生成低秩适配（LoRA）的超网络方法，利用用户上下文直接合成个性化适配器，兼具推理时仅需前向传播和模型权重更新的优势。

用在哪里

适用于在手机等资源受限设备上提供定制化对话或文本生成的服务场景，对隐私要求高、希望减少云端依赖的开发者与研究者会感兴趣。

可以推断的

推测：超网络在部署前需要在大规模用户上下文数据上进行离线训练，这一步的计算成本可能较大。
推测：由于适配器在本地生成，若模型或语言能力需要更新，可能要重新训练或微调超网络本身。

来源摘要/节选

On-device large language models (LLMs'), e.g. running on mobile phones, are ripe for improvement via personalization. The limited compute resources of mobile devices impose limits on model scale and thus model quality, making any realizable quality gains highly impactful. At the same time, their personal nature (i.e., the close coupling to a particular user) means that a given on-device LLM tends to be used in similar, predictable patterns over the course of time. This paper presents a novel method for personalizing on-device LLMs. It trains a hypernetwork to map a user's context tokens to a low-rank adaptation (LoRA’) well-suited to that user. Once the trained common artifacts are deployed to users’ devices, each user uses the hypernetwork to synthesize (entirely on device) a personalized LoRA. This approach blends the benefits while avoiding the drawbacks of two existing approaches to LLM customization: in-context learning (ICL') and parameter-efficient fine-tuning (PEFT’). Like ICL (and unlike PEFT), the on-device phase of our approach is computationally feasible, requiring only forward passes through neural networks. Like PEFT (and unlike ICL), our approach modifies the `target’ base LLM via weights (the LoRA), avoiding negative consequences (e.g. increased latency) associated with extending the input sequence. Our approach is particularly well-suited to the mobile device regime. Apart from the on-device compute and latency benefits mentioned, it also requires minimal additional storage, as internally its architecture partly leverages the same LLM weights as belong to the target LLM to be personalized. We demonstrate the benefits of LoRA-generating hypernetworks on several representative personalization datasets, comparing against baselines like ICL and PEFT. Of note, our personalization experiments focus on more challenging and less studied long-form text generation tasks.

来源说明

当前保存的是来源摘要，不代表论文全文。请以原始来源为准。

「要点解读」由 AI Stack 依据上方已保存内容整理，不代表来源的完整表述；标注「推测：」的判断来自编辑，不是来源陈述。