基本信息
- 来源: arxiv
- 原始来源: http://arxiv.org/abs/2608.06361v1
- 发布域名: arxiv.org
- 分类: cs.AI
- 作者: Sarvesh Baskar、Zikui Cai、Shayan Shabihi 等
要点解读
这是什么
这是一篇关于视频语言模型在计数、频率和状态转换等简单事件上表现的评估研究，提出基于可执行事件轨迹的分析方法，揭示模型在时序信息处理上的阶段性失效。
用在哪里
适用于视频模型评测框架的设计者和对时序推理能力有需求的开发者，尤其在需要区分事件持久性与瞬时性的任务中。
可以推断的
推测：在此类基准上，模型对瞬时事件的感知往往比持续状态更受限。
推测：提升采样率可能提升计数精度，但并不等价于事件序列的真实恢复。
来源摘要/节选
Real-world video benchmarks provide broad coverage, but their fixed clips entangle event count, rate, duration, and visual complexity, making failure modes hard to isolate. While existing programmatic benchmarks offer better control, they score only the final answer rather than auditing reported events against executable ground truth. To bridge this gap, we introduce trace-grounded parametric profiling for event counting in three controlled video tasks: bouncing-ball wall contacts, visual blinks, and categorical state transitions. Across 2,190 videos, we vary event count N and frequency F while holding rendering fixed. Each video includes an executable event trace for capability-surface estimation and timestamp-level evaluation. Our results reveal a staged temporal failure. At an 80% reliability threshold, Gemini 3.6 Flash reliably counts persistent state transitions up to 12 events at 0.5 and 1.0 Hz, yet demonstrates no reliable positive-count region for transient blinking events. Thus, event representation dictates whether a model initially accesses evidence – a limitation that compounds as count and frequency increase. In the high-count, high-frequency regime, only 0.2% of final counts are correct and the model recovers just 18.1% of true events. To test if visual access is the primary bottleneck, we increase sampling rate. Although this boosts Bounce Ball accuracy from 19.6% to 29.3%, the reported sequence agrees with ground truth only 3.7% of the time. Extra frames can therefore inflate final scores without producing faithful event recovery. Different prompting strategies yield similarly limited gains, and real-world video evaluations show the same concentration of success at low event counts. Ultimately, trace-grounded profiling shifts video evaluation from aggregate accuracy metrics to a detailed diagnostic of where temporal reasoning fails.
来源说明
当前保存的是来源摘要，不代表论文全文。请以原始来源为准。
「要点解读」由 AI Stack 依据上方已保存内容整理，不代表来源的完整表述；标注「推测：」的判断来自编辑，不是来源陈述。