液态绘图:利用变化式自动电解器生成不限 (Latent Mappings: Generating Open-Ended Expressive Mappings Using Variational Autoencoders)

from arxiv, Published at the International Conference on New Interfaces for Musical Expression, June 2021. 3000 word short paper. 5 figures plus video which may be seen at https://timmb.com/sonified-body-r-and-d-lab

In many contexts, creating mappings for gestural interactions can form part of an artistic process. Creators seeking a mapping that is expressive, novel, and affords them a sense of authorship may not know how to program it up in a signal processing patch. Tools like Wekinator and MIMIC allow creators to use supervised machine learning to learn mappings from example input/output pairings. However, a creator may know a good mapping when they encounter it yet start with little sense of what the inputs or outputs should be. We call this an open-ended mapping process. Addressing this need, we introduce the latent mapping, which leverages the latent space of an unsupervised machine learning algorithm such as a Variational Autoencoder trained on a corpus of unlabelled gestural data from the creator. We illustrate it with Sonified Body, a system mapping full-body movement to sound which we explore in a residency with three dancers.

翻译：在许多情形下,为视觉互动创建绘图可以构成艺术过程的一部分。创造者寻求一种表达式、新颖的、能给他们带来一种作者感的映射,他们可能不知道如何在信号处理补丁中进行编程。 Wekinator 和 MIMIC 等工具允许创作者使用受监督的机器学习,从示例输入/产出配对中学习绘图。但是, 创作者在遇到时可能知道一个良好的映射过程, 但却没有多少关于投入或产出应该是什么的感知。我们称之为一个开放式的映射过程。满足这一需要, 我们引入了潜在映射, 从而利用一个不受监督的机器学习算法的潜在空间, 比如, 由创作者训练的一组无标签的基因数据。我们用Sonific body来演示它, 一个系统, 用来绘制我们与三名舞者在住所中探索的完整身体运动。

相关内容

自编码器

关注 140

自动编码器是一种人工神经网络，用于以无监督的方式学习有效的数据编码。自动编码器的目的是通过训练网络忽略信号“噪声”来学习一组数据的表示（编码），通常用于降维。与简化方面一起，学习了重构方面，在此，自动编码器尝试从简化编码中生成尽可能接近其原始输入的表示形式，从而得到其名称。基本模型存在几种变体，其目的是迫使学习的输入表示形式具有有用的属性。自动编码器可有效地解决许多应用问题，从面部识别到获取单词的语义。