Cross-topic stance detection is the task to automatically detect stances (pro, against, or neutral) on unseen topics. We successfully reproduce state-of-the-art cross-topic stance detection work (Reimers et. al., 2019), and systematically analyze its reproducibility. Our attention then turns to the cross-topic aspect of this work, and the specificity of topics in terms of vocabulary and socio-cultural context. We ask: To what extent is stance detection topic-independent and generalizable across topics? We compare the model's performance on various unseen topics, and find topic (e.g. abortion, cloning), class (e.g. pro, con), and their interaction affecting the model's performance. We conclude that investigating performance on different topics, and addressing topic-specific vocabulary and context, is a future avenue for cross-topic stance detection.
翻译:跨主题定位探测是自动检测关于无形议题的立场(pro, 反对或中性)的任务。 我们成功地复制了最先进的跨主题定位探测工作(Reimers等人, 2019年),并系统地分析其可复制性。 我们然后将注意力转向这项工作的跨主题方面,以及词汇和社会文化背景方面专题的特殊性。 我们问: 立场检测专题独立和可在各个专题之间普及的程度是多少? 我们比较模型在各种不可见专题上的绩效,并发现专题(例如堕胎、克隆)、类(例如赞成、反对)和影响模型绩效的相互作用。 我们的结论是,调查不同专题的业绩,处理特定专题的词汇和背景,是今后跨主题定位检测的一个途径。