3D hand shape and pose estimation from a single depth map is a new and challenging computer vision problem with many applications. Existing methods addressing it directly regress hand meshes via 2D convolutional neural networks, which leads to artifacts due to perspective distortions in the images. To address the limitations of the existing methods, we develop HandVoxNet++, i.e., a voxel-based deep network with 3D and graph convolutions trained in a fully supervised manner. The input to our network is a 3D voxelized-depth-map-based on the truncated signed distance function (TSDF). HandVoxNet++ relies on two hand shape representations. The first one is the 3D voxelized grid of hand shape, which does not preserve the mesh topology and which is the most accurate representation. The second representation is the hand surface that preserves the mesh topology. We combine the advantages of both representations by aligning the hand surface to the voxelized hand shape either with a new neural Graph-Convolutions-based Mesh Registration (GCN-MeshReg) or classical segment-wise Non-Rigid Gravitational Approach (NRGA++) which does not rely on training data. In extensive evaluations on three public benchmarks, i.e., SynHand5M, depth-based HANDS19 challenge and HO-3D, the proposed HandVoxNet++ achieves the state-of-the-art performance. In this journal extension of our previous approach presented at CVPR 2020, we gain 41.09% and 13.7% higher shape alignment accuracy on SynHand5M and HANDS19 datasets, respectively. Our method is ranked first on the HANDS19 challenge dataset (Task 1: Depth-Based 3D Hand Pose Estimation) at the moment of the submission of our results to the portal in August 2020.


翻译:3D 手形和从单一深度映射中作出估计是一个具有挑战性的新型计算机图像问题。 许多应用程序都存在。 通过 2D 进化神经网络直接递退手模件的现存方法, 这导致图像的视觉扭曲。 为解决现有方法的局限性, 我们开发HandVoxNet+++, 即基于 voxel 的深网络, 由 3D 和 图形的完全监控方式培训。 我们网络的输入是 3D 驱动的 计算机门户网站 。 以 快速签名远程功能( TSDF) 为基础, 3DVelel 深度的计算机图像。 HandVoxNet++ 依赖两个手形表达方式。 第一个是 3D vell化的手形网格, 它不会保存网形图示, 也就是以 3DVD为主的 。 我们的手表面与基于 Scial- 3Server 的图像转换方法( GCN-Mesh- Reeg+++) 显示两个直径直径直径直径, 的H- deal- dal- disal- destal- disal- disal- disal- disal- disal- disal- dismal- disal- disl- dismal- disl) 。 在三 Reval- disal- disal- disal- disal- disal- dismal- dismal- disfal- disfal- dismal- disgal- 上, 上, 上, 上,,,, 和O- dis- dis- dis- dis- dis- disg- dis- disl- disg- disgal- disgald- disgal- dism- disgald- disald- disald-d-d-d-d- dism- disal- disal-d- disal- disal- dism- disl- dism- dism- 和O- 和O- 上, 上,

0
下载
关闭预览

相关内容

iOS 8 提供的应用间和应用跟系统的功能交互特性。
  • Today (iOS and OS X): widgets for the Today view of Notification Center
  • Share (iOS and OS X): post content to web services or share content with others
  • Actions (iOS and OS X): app extensions to view or manipulate inside another app
  • Photo Editing (iOS): edit a photo or video in Apple's Photos app with extensions from a third-party apps
  • Finder Sync (OS X): remote file storage in the Finder with support for Finder content annotation
  • Storage Provider (iOS): an interface between files inside an app and other apps on a user's device
  • Custom Keyboard (iOS): system-wide alternative keyboards

Source: iOS 8 Extensions: Apple’s Plan for a Powerful App Ecosystem
专知会员服务
31+阅读 · 2021年6月12日
【图神经网络导论】Intro to Graph Neural Networks,176页ppt
专知会员服务
125+阅读 · 2021年6月4日
【图与几何深度学习】Graph and geometric deep learning,49页ppt
CVPR2020 | 商汤-港中文等提出PV-RCNN:3D目标检测新网络
专知会员服务
43+阅读 · 2020年4月17日
专知会员服务
109+阅读 · 2020年3月12日
抢鲜看!13篇CVPR2020论文链接/开源代码/解读
专知会员服务
49+阅读 · 2020年2月26日
“CVPR 2020 接受论文列表 1470篇论文都在这了
Hierarchically Structured Meta-learning
CreateAMind
26+阅读 · 2019年5月22日
TCN v2 + 3Dconv 运动信息
CreateAMind
4+阅读 · 2019年1月8日
【跟踪Tracking】15篇论文+代码 | 中秋快乐~
专知
18+阅读 · 2018年9月24日
【推荐】(TensorFlow)SSD实时手部检测与追踪(附代码)
机器学习研究会
11+阅读 · 2017年12月5日
Capsule Networks解析
机器学习研究会
11+阅读 · 2017年11月12日
Voxel Transformer for 3D Object Detection
Arxiv
1+阅读 · 2021年9月6日
Arxiv
27+阅读 · 2020年12月24日
VIP会员
Top
微信扫码咨询专知VIP会员