This paper presents the development of a multi-sensor user interface to facilitate the instruction of arc welding tasks. Traditional methods to acquire hand-eye coordination skills are typically conducted through one-to-one instruction where trainees must wear protective helmets and conduct several tests. This approach is inefficient as the harmful light emitted from the electric arc impedes the close monitoring of the process; Practitioners can only observe a small bright spot. To tackle these problems, recent training approaches have leveraged virtual reality to safely simulate the process and visualize the geometry of the workpieces. However, the synthetic nature of these types of simulation platforms reduces their effectiveness as they fail to comprise actual welding interactions with the environment, which hinders the trainees' learning process. To provide users with a real welding experience, we have developed a new multi-sensor extended reality platform for arc welding training. Our system is composed of: (1) An HDR camera, monitoring the real welding spot in real-time; (2) A depth sensor, capturing the 3D geometry of the scene; and (3) A head-mounted VR display, visualizing the process safely. Our innovative platform provides users with a "bot trainer", virtual cues of the seam geometry, automatic spot tracking, and performance scores. To validate the platform's feasibility, we conduct extensive experiments with several welding training tasks. We show that compared with the traditional training practice and recent virtual reality approaches, our automated multi-sensor method achieves better performances in terms of accuracy, learning curve, and effectiveness.
翻译:本文介绍了多传感器用户界面的开发情况,以方便电弧焊接任务教学。获取手眼协调技能的传统方法通常通过一对一的教学进行,受训人员必须佩戴防护头盔并进行若干测试。这种方法效率低下,因为电弧发出的有害光阻碍对流程的密切监测;从业者只能观察一个小亮点。为了解决这些问题,最近的培训方法利用虚拟现实安全模拟流程并直观工作机件的几何结构。然而,这些模拟平台的合成性质降低了其有效性,因为它们不能包括实际与环境的焊接性互动,从而阻碍受训人员学习过程。为用户提供真正的焊接经验,我们开发了一个新的多传感器扩展的现实平台,用于焊接培训。我们的系统由以下组成:(1) 人类发展报告相机,实时监测多个实际焊接点;(2) 深度传感器,捕捉现场的3D几何测量;(3) 头置VR显示甚高的虚拟平台,以安全化的模拟操作方式对流程进行可视化分析,我们的创新平台为用户提供了一种模拟性能展示,同时显示我们实地的实地训练方法。