Illustrations are widely used in education, and sometimes, alternatives are not available for visually impaired students. Therefore, those students would benefit greatly from an automatic illustration description system, but only if those descriptions were complete, correct, and easily understandable using a screenreader. In this paper, we report on a study for the assessment of automated image descriptions. We interviewed experts to establish evaluation criteria, which we then used to create an evaluation questionnaire for sighted non-expert raters, and description templates. We used this questionnaire to evaluate the quality of descriptions which could be generated with a template-based automatic image describer. We present evidence that these templates have the potential to generate useful descriptions, and that the questionnaire identifies problems with description templates.
翻译:说明在教育中广泛使用,有时对低视力学生没有替代方法,因此,这些学生将大大受益于自动插图描述系统,但前提是这些说明是完整、正确和容易理解的,使用屏幕阅读器。在本文件中,我们报告对自动图像描述的评估研究情况。我们约谈专家以制定评价标准,然后我们用这些标准为有视力的非专家评级员编制评价问卷,并使用描述模板。我们利用这一问卷评估用模板自动图像描述器生成描述的质量。我们提供证据表明,这些模板有可能产生有用的描述,调查问卷发现描述模板存在问题。