基本信息
- 来源: arxiv
- 原始来源: http://arxiv.org/abs/2607.29677v1
- 发布域名: arxiv.org
- 分类: cs.AI
- 作者: Boyang Zhang、Adrian Lyjak、Eli Stewart 等
要点解读
这是什么
这是一个针对企业文档的模式引导提取任务的基准测试，用于评估系统在值准确性、记录完整性、溯源可信度以及运行成本等多个维度的表现。
用在哪里
适合企业在搭建自动文档处理流程时对比不同模型的实际效果，也适合研究人员检验新方法在大规模真实文档上的性能。
可以推断的
推测：在实际业务中，处理页数较多的文档时，系统可能出现信息截断或遗漏记录的情况。
推测：企业在选择提取方案时，除了关注准确率外，运行成本也会成为重要考量因素。
来源摘要/节选
Enterprise workflows increasingly rely on agents for \emph{schema-guided extraction}: given a document and a user-defined schema, the agent faithfully follows the schema to produce the correct output with source evidence as grounding metadata. We present ExtractBench, a benchmark for schema-guided extraction and, to our knowledge, the first to score value accuracy, record completeness at scale, grounding, and measured cost together. The evaluation system contains 4,869 pages across 370 enterprise documents, 8 business domains, and 67 document types, with clear tags differentiating their challenge scenarios. The scalable schema and ground-truth curation pipeline combines independent-system agreement for real documents, known values for synthetic lists, and human verification for forms. We report order-insensitive value F1 for value accuracy, plus two grounding metrics for source traceability: word- and page-level F1. Commercial VLMs perform well on short documents but often truncate record lists on long ones, while coding agents retain higher accuracy at much higher cost. LlamaExtract Agentic Plus ranks first on all three metrics, with accuracy comparable to coding agents at a fraction of the cost. Dataset and evaluation code are available on \href{https://huggingface.co/datasets/llamaindex/ExtractBench}{HuggingFace} and \href{https://github.com/run-llama/ExtractBench}{GitHub}.
来源说明
当前保存的是来源摘要，不代表论文全文。请以原始来源为准。
「要点解读」由 AI Stack 依据上方已保存内容整理，不代表来源的完整表述；标注「推测：」的判断来自编辑，不是来源陈述。