Recent works show that interactive documents connecting text with visualizations facilitate reading comprehension. However, creating this type of content requires specialized knowledge. We present ChartText, a method that links text with visualizations in this work. Our approach supports documents that include bar charts, line charts, and scatter plots. ChartText receives the visual encoding of the visualization and its associated text as input. It then performs the linking in two stages: The matching stage creates individual links relating simple phrases between the text and the chart. Then, it combines the individual links according to the visual channels in the grouping stage, building more meaningful connections. We use two datasets to design and evaluate our method; the first comes from web documents (24 bar charts and texts) and the second from academic documents (25 bar charts, 25 line charts, and 25 scatter plots with their texts). Our experiments show that our method obtains F1 scores of 0.50 and 0.66 on both datasets. We can also use a semi-automatic approach correcting individual links; in this case, the scores rise to 0.68 and 0.84, respectively. To show the usefulness of our technique, we implement two proofs of concept. We create interactive documents using graphic overlays in the first one, facilitating the reading experience. We use voice instead of text to annotate charts in real-time in the second. For example, in a videoconference, our technique can automatically annotate a chart following the presenter's description.
翻译:最近的工作显示,将文本与可视化相联系的交互式文档有助于阅读理解。然而,创建这种类型的内容需要专业知识。我们提供图表图文本,这是将文本与这项工作中的可视化联系起来的一种方法。我们的方法支持包括条形图表、行图图和散射图在内的文件。图图图文本以输入方式获得可视化及其相关文本的视觉编码。然后,它分两个阶段进行连接:匹配阶段创建了与文本和图表之间简单短语相关的单个链接。然后,根据组合阶段的视觉渠道将个人链接合并起来,建立更有意义的连接。我们用两个数据集来设计和评估我们的方法;第一个来自网络文件(24条图和文本),第二个来自学术文件(25条条形图表、25条线图和25条散射图及其文本)。我们的实验显示,我们的方法在两个数据集上都获得了F1分0.50和0.66分。我们还可以使用半自动的方法纠正个人链接;在这种情况下,分数会上升到0.68和0.84。为了显示我们的第一个技术的效用,我们用两个技术,我们用互动的图表,我们用一个图表,我们用一个图表,我们用一个图表来自动的图表,我们用一个图表,我们用一个图表,我们用一个图表来做一个图表。我们用一个图表来解释。我们用一个图。我们用一个图的图。我们用一个图。我们用一个图。我们用一个图。我们用一个图。我们用一个图。