This paper introduces Sketched Reality, an approach that combines AR sketching and actuated tangible user interfaces (TUI) for bidirectional sketching interaction. Bi-directional sketching enables virtual sketches and physical objects to "affect" each other through physical actuation and digital computation. In the existing AR sketching, the relationship between virtual and physical worlds is only one-directional -- while physical interaction can affect virtual sketches, virtual sketches have no return effect on the physical objects or environment. In contrast, bi-directional sketching interaction allows the seamless coupling between sketches and actuated TUIs. In this paper, we employ tabletop-size small robots (Sony Toio) and an iPad-based AR sketching tool to demonstrate the concept. In our system, virtual sketches drawn and simulated on an iPad (e.g., lines, walls, pendulums, and springs) can move, actuate, collide, and constrain physical Toio robots, as if virtual sketches and the physical objects exist in the same space through seamless coupling between AR and robot motion. This paper contributes a set of novel interactions and a design space of bi-directional AR sketching. We demonstrate a series of potential applications, such as tangible physics education, explorable mechanism, tangible gaming for children, and in-situ robot programming via sketching.
翻译:本文介绍Sketed Reality, 这是一种将AR 素描和活化的有形用户界面(TUI)结合成双向素描互动的方法。 双向素描使虚拟素描和物理物体能够通过物理振动和数字计算“ 相互影响 ” 。 在现有的 AR 素描中,虚拟世界和物理世界之间的关系只能是一向的, 而物理互动可以影响虚拟素描, 虚拟素描不会对物理物体或环境产生回响效应。 相反, 双向素描互动可以让素描和活性图解介之间无缝的结合。 在这个纸上, 我们使用桌面大小的小机器人(Sony Toio)和基于iPad的真像素描写工具来展示这个概念。 在我们的系统中, 虚拟素描画和模拟的虚拟画像(如线、墙壁、笔画和弹簧)可以移动、 模拟和限制物理托奥机器人, 就像虚拟素描一样, 我们的物理物体通过机器和虚拟的图谱应用, 通过一个无缝的图像设计空间设计 展示, 展示一个机器人和虚拟空间的模型, 展示, 我们的模型, 展示, 的模型的模型的模型的模型的模型的模型和空间的模型的模型的模型的模型的模型的模型的模型的模型, 展示, 展示, 和空间的模型的模型的模型的模型的模型的模型的模型的机制成。