Hierarchical Path-planning from Speech Instructions with Spatial Concept-based Topometric Semantic Mapping

Navigating to destinations using human speech instructions is essential for autonomous mobile robots operating in the real world. Although robots can take different paths toward the same goal, the shortest path is not always optimal. A desired approach is to flexibly accommodate waypoint specifications, planning a better alternative path, even with detours. Furthermore, robots require real-time inference capabilities. Spatial representations include semantic, topological, and metric levels, each capturing different aspects of the environment. This study aims to realize a hierarchical spatial representation by a topometric semantic map and path planning with speech instructions, including waypoints. We propose SpCoTMHP, a hierarchical path-planning method that utilizes multimodal spatial concepts, incorporating place connectivity. This approach provides a novel integrated probabilistic generative model and fast approximate inference, with interaction among the hierarchy levels. A formulation based on control as probabilistic inference theoretically supports the proposed path planning. Navigation experiments using speech instruction with a waypoint demonstrated the performance improvement of path planning, WN-SPL by 0.589, and reduced computation time by 7.14 sec compared to conventional methods. Hierarchical spatial representations offer a mutually understandable form for humans and robots, enabling language-based navigation tasks.