Algorithms for reliable real-time score following in live opera promise a lot of useful applications such as automatic subtitles display, or real-time video cutting in live streaming. Until now, such systems were based on the strong assumption that an opera performance follows the structure of the score linearly. However, this is rarely the case in practice, because of different opera versions and directors' cutting choices. In this paper, we propose a two-level solution to this problem. We introduce a real-time-capable, high-resolution (HR) tracker that can handle jumps or repetitions at specific locations provided to it. We then combine this with an additional low-resolution (LR) tracker that can handle all sorts of mismatches that can occur at any time, with some imprecision, and can re-direct the HR tracker if the latter is `lost' in the score. We show that the combination of the two improves tracking robustness in the presence of strong structural mismatches.
翻译：在现场歌剧中,可靠实时分数的比喻在现场歌剧中可以保证许多有用的应用,例如自动字幕显示或实时视频在现场流中切换。直到现在,这些系统都基于歌剧表演遵循线性评分结构的强烈假设。然而,由于歌剧版本和导演的切分选择不同,在实践中这种情况很少发生。在本文中,我们提出了解决这一问题的双重解决方案。我们引入了一个实时、高分辨率的追踪器,可以处理特定地点提供的跳动或重复。我们随后将它与另外一种低分辨率追踪器结合起来,它可以处理随时可能发生的各种不匹配,有些不精确,如果HR追踪器在评分中“丢失 ”, 并且如果后者在分数中“ 丢失 ”, 就可以对人力资源追踪器进行再定向。我们显示,在存在强大的结构不匹配的情况下,这两种方法的组合可以改进对稳健性的跟踪。
相关内容
微信扫码咨询专知VIP会员