基本信息
- 来源: arxiv
- 原始来源: http://arxiv.org/abs/2608.27424v1
- 发布域名: arxiv.org
- 分类: cs.CR
- 作者: Qianlong Lan、Vinothini Pandurangan、Anuj Kaul 等
要点解读
这是什么
该研究利用合成的 Pickle 与 PyTorch 样本库，对比了 ModelScan、ModelAudit 与 Fickling 在机器学习模型安全检查中的判断覆盖率、能否给出确定性结论以及准确性，并分析了工具之间的冗余情况。
用在哪里
适用于在机器学习部署流程中需要挑选或评估静态安全扫描工具的团队，也适合对现有扫描方案进行基准测试和冗余分析的安全研究人员。
可以推断的
推测：在实际项目中，仅依赖单一工具的覆盖率可能遗漏部分恶意模型的风险。
推测：工具组合使用可以在覆盖率上互补，尤其在某些工具分析失败时，其他工具仍可能提供可靠的检测结果。
来源摘要/节选
Static scanners are increasingly used to identify executable or otherwise unsafe content in machine- learning artifacts, yet conventional evaluation metrics characterize only cases where a scanner yields a usable security judgment. We evaluate ModelScan, ModelAudit, and Fickling using a controlled, artifact-backed benchmark on a synthetic corpus of 170 Pickle and PyTorch focused artifacts across 145 specimen families, 135 of which have binary security ground truth and 10 of which are intentionally malformed without labels. We explicitly distinguish non-N/A coverage, analysis completion, definitive security decisions, non-security findings, and unsupported outcomes. On labeled families, ModelAudit produced definitive security decisions for all 135 families (100%), Fickling for 110 (81.5%), and ModelScan for 67 (49.6%). Conditional on making a definitive judgment, ModelScan achieved 100% precision, recall, and F1. Fickling identified no unique true- positive families beyond those found by the combination of ModelAudit and ModelScan. Furthermore, for the 48 malicious families where ModelScan failed to complete its analysis, both ModelAudit and Fickling generated detections consistent with ground truth. These findings underscore the need to separate judgment accuracy from judgment availability, as well as incremental detection coverage from tool-level redundancy.
来源说明
当前保存的是来源摘要，不代表论文全文。请以原始来源为准。
「要点解读」由 AI Stack 依据上方已保存内容整理，不代表来源的完整表述；标注「推测：」的判断来自编辑，不是来源陈述。