机器学习系统设计系统评估标准

VIP内容

论文题目

Model Cards for Model Reporting

论文摘要

训练有素的机器学习模式越来越多地用于执行执法、医学、教育和就业等领域的高影响力任务。为了澄清机器学习模型的预期用例,并尽量减少它们在不太适合的上下文中的使用,我们建议发布的模型附带详细说明其性能特征的文档。在本文中,我们提出了一个框架,我们称之为模型卡,以鼓励这种透明的模型报告。模型卡是经过培训的机器学习模型附带的简短文档,在各种条件下提供基准评估,例如跨不同文化、人口或表型群体(例如种族、地理位置、性别、Fitzpatrick皮肤类型)和跨部门群体(例如年龄和种族,或者性别和菲茨帕特里克皮肤类型)与预期应用领域相关。模型卡还披露了模型的使用环境、性能评估程序的细节以及其他相关信息。虽然我们主要关注以人为中心的机器学习模型在计算机视觉和自然语言处理领域的应用,但是这个框架可以用来记录任何经过训练的机器学习模型。为了巩固这一概念,我们为两种监督模式提供卡片:一种是训练来检测图像中的笑脸,另一种是训练来检测文本中的有毒评论。我们建议将模型卡作为机器学习和相关人工智能技术负责任民主化的一个步骤,提高人工智能技术如何工作的透明度。我们希望这项工作能够鼓励那些发布经过培训的机器学习模型的人在发布模型时附带类似的详细评估数字和其他相关文档。

论文作者

玛格丽特·米切尔、西蒙妮·吴、安德鲁·扎尔迪瓦尔、帕克·巴恩斯、露西·瓦瑟曼、本·哈钦森、埃琳娜·斯皮策、伊诺鲁瓦·德博拉·拉吉、蒂姆尼·格布鲁,来自google人工智能团队。

成为VIP会员查看完整内容
0
9

最新内容

Semantic Scene Completion aims at reconstructing a complete 3D scene with precise voxel-wise semantics from a single-view depth or RGBD image. It is a crucial but challenging problem for indoor scene understanding. In this work, we present a novel framework named Scene-Instance-Scene Network (\textit{SISNet}), which takes advantages of both instance and scene level semantic information. Our method is capable of inferring fine-grained shape details as well as nearby objects whose semantic categories are easily mixed-up. The key insight is that we decouple the instances from a coarsely completed semantic scene instead of a raw input image to guide the reconstruction of instances and the overall scene. SISNet conducts iterative scene-to-instance (SI) and instance-to-scene (IS) semantic completion. Specifically, the SI is able to encode objects' surrounding context for effectively decoupling instances from the scene and each instance could be voxelized into higher resolution to capture finer details. With IS, fine-grained instance information can be integrated back into the 3D scene and thus leads to more accurate semantic scene completion. Utilizing such an iterative mechanism, the scene and instance completion benefits each other to achieve higher completion accuracy. Extensively experiments show that our proposed method consistently outperforms state-of-the-art methods on both real NYU, NYUCAD and synthetic SUNCG-RGBD datasets. The code and the supplementary material will be available at \url{https://github.com/yjcaimeow/SISNet}.

0
0
下载
预览

最新论文

Semantic Scene Completion aims at reconstructing a complete 3D scene with precise voxel-wise semantics from a single-view depth or RGBD image. It is a crucial but challenging problem for indoor scene understanding. In this work, we present a novel framework named Scene-Instance-Scene Network (\textit{SISNet}), which takes advantages of both instance and scene level semantic information. Our method is capable of inferring fine-grained shape details as well as nearby objects whose semantic categories are easily mixed-up. The key insight is that we decouple the instances from a coarsely completed semantic scene instead of a raw input image to guide the reconstruction of instances and the overall scene. SISNet conducts iterative scene-to-instance (SI) and instance-to-scene (IS) semantic completion. Specifically, the SI is able to encode objects' surrounding context for effectively decoupling instances from the scene and each instance could be voxelized into higher resolution to capture finer details. With IS, fine-grained instance information can be integrated back into the 3D scene and thus leads to more accurate semantic scene completion. Utilizing such an iterative mechanism, the scene and instance completion benefits each other to achieve higher completion accuracy. Extensively experiments show that our proposed method consistently outperforms state-of-the-art methods on both real NYU, NYUCAD and synthetic SUNCG-RGBD datasets. The code and the supplementary material will be available at \url{https://github.com/yjcaimeow/SISNet}.

0
0
下载
预览
父主题
Top