{"id":799,"date":"2021-12-02T21:13:54","date_gmt":"2021-12-02T13:13:54","guid":{"rendered":"https:\/\/511cvlab.sinkers.cn\/?p=799"},"modified":"2025-10-17T16:58:09","modified_gmt":"2025-10-17T08:58:09","slug":"phg","status":"publish","type":"post","link":"https:\/\/cv.nirc.top\/zh\/2021\/phg\/","title":{"rendered":"Pose-Guided Hierarchical Graph Reasoning for 3-D Hand Pose Estimation From a Single Depth Image"},"content":{"rendered":"
\n
\n \n
\n \n
\n \n \n <\/path>\n <\/svg>\n <\/div>\n
Paper<\/div>\n <\/a>\n <\/div>\n\n \n\n\n <\/div>\n<\/div>\n\n\n\n

<\/p>\n\n\n\n

\"\"\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n\n\n\n

Due to the self-similarity of the fingers and severe self-occlusion, it is difficult to predict the correct joint position from local evidence (as shown in Fig. a). By incorporating context information (as shown in Fig. b, where adjacent joints can be accurately predicted), forming enhanced feature maps through pose-guided hierarchical graph (PHG)<\/strong>, ambiguity is effectively reduced (as shown in Fig. c).<\/p>\n\n\n\n

Abstract<\/h2>\n\n\n\n

Estimating 3-D hand pose estimation from a single depth image is important for human\u2013computer interaction. Although depth-based 3-D hand pose estimation has made great progress in recent years, it is still difficult to deal with some complex scenes, especially the issues of serious self-occlusion and high self-similarity of fingers. Inspired by the fact that multipart context is critical to alleviate ambiguity, and constraint relations contained in the hand structure are important for robust estimation, we attempt to explicitly model the correlations between different hand parts. In this article, we propose a pose-guided hierarchical graph convolution (PHG) module, which is embedded into the pixelwise regression framework to enhance the convolutional feature maps by exploring the complex dependencies between different hand parts. Specifically, the PHG module first extracts hierarchical fine-grained node features under the guidance of hand pose and then uses graph convolution to perform hierarchical message passing between nodes according to the hand structure. Finally, the enhanced node features are used to generate dynamic convolution kernels to generate hierarchical structure-aware feature maps. Our method achieves state-of-the-art performance or comparable performance with the state-of-the-art methods on five 3-D hand pose datasets: HANDS 2019, HANDS 2017, NYU, ICVL, and MSRA\u200b.<\/p>\n\n\n\n

Overview<\/h2>\n\n\n\n
\"\"\/\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button>
Network architecture. (a)
Overall framework<\/strong> of our method. (b) Detailed composition of the pose-guided hierarchical graph (PHG)<\/strong> module. Composed of node mapping<\/strong> module, residual graph convolution<\/strong> module, and node remapping<\/strong> modele. (c) Detailed composition of detection-based head (DH) <\/strong>module.<\/figcaption><\/figure>\n\n\n\n

<\/p>\n\n\n\n

1\ufe0f\u20e3 A PFE module extracts the initial feature maps from the depth image.
2\ufe0f\u20e3 The backbone adopts an encoding-decoding structure to generate the feature maps with high-level semantics.
3\ufe0f\u20e3 The PHG<\/strong> module generates enhanced feature maps<\/strong> by utilizing dependency among different hand parts.
4\ufe0f\u20e3 The DH<\/strong> module is used to perform the pixelwise estimation<\/strong>. In particular, adding a DH after the backbone is able to provide an auxiliary loss<\/strong> to reduce the difficulty of network training.<\/p>\n\n\n\n

Our network can also contain multiple stages. If network stacking<\/strong> is performed, as shown in stage 2, two 1\u00d71 convolutions, respectively, remap the upsampled pixelwise estimations and the upsampled enhanced feature maps to the intermediate features, which are added with initial feature maps and fed to the subsequent subnetwork.<\/p>\n\n\n\n

Qualitative result<\/h2>\n\n\n\n

<\/p>\n\n\n\n

\"\"\/\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button>
Qualitative results for HANDS 2017 and HANDS 2019 datasets. Left: Qualitative results of the images with self-similarity and self-occlusion. Right: Failure cases. For
self-occlusion and self-similarity<\/strong>, our method obtains more accurate and reasonable poses than the strong baseline (AWR). <\/figcaption><\/figure>\n\n\n\n

<\/p>\n\n\n\n

Comparison with SOTA<\/h2>\n\n\n\n
\"\"\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button>
Comparison of the mean joint error (mm) on NYU, ICVL, and MSRA datasets. Methods with \u2217 crop the hand using the ground truth joint. Methods with \u2020 exploit temporal information. Methods with \u2021 use additional data such as multiview images or synthetic images\u200b.<\/figcaption><\/figure>\n\n\n\n
\"\"\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button>
Comparison with state-of-the-art methods on HANDS 2019\u200b.<\/figcaption><\/figure>\n\n\n\n
\"\"\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button>
Comparison with state-of-the-art methods on HANDS 2017. AVG represents the mean joint error (mm) across the test set. SEEN and UNSEEN indicate whether the subjects to which the tested samples belonged had ever appeared in the training set.<\/figcaption><\/figure>\n\n\n\n

Conclusion<\/h2>\n\n\n\n

We introduced a PHG Network for robust and accurate 3-D hand pose estimation. Our method was embedded in the pixelwise regression framework<\/strong>, which can maintain the spatial structure<\/strong> of feature maps while capturing the complex short and long-range dependency<\/strong> of different hand regions. We proposed the following:<\/p>\n\n\n\n

1\ufe0f\u20e3 A node mapping module<\/strong> to extract fine-grained node features, guided by the hand pose information.
2\ufe0f\u20e3 The residual graph convolution module<\/strong>, which performs iterative and hierarchical message passing. This module is able to capture the complex dependency relations between joints and bones.
3\ufe0f\u20e3 The enhanced node features are dynamically encoded into a set of convolution kernels throuh node remapping module<\/strong> to generate hierarchical structure-aware feature maps, which can be used for more accurate and robust hand pose estimation.<\/p>\n\n\n\n

Our method is particularly effective in handling cases with severe self-similarity and self-occlusion. Extensive experiments on five public hand datasets demonstrated the effectiveness of our approach.<\/p>\n\n\n\n

Bibtex<\/h2>\n\n\n\n
@article{ren2021pose,\n  title={Pose-guided hierarchical graph reasoning for 3-d hand pose estimation from a single depth image},\n  author={Ren, Pengfei and Sun, Haifeng and Hao, Jiachang and Qi, Qi and Wang, Jingyu and Liao, Jianxin},\n  journal={IEEE Transactions on Cybernetics},\n  volume={53},\n  number={1},\n  pages={315--328},\n  year={2021},\n  publisher={IEEE}\n}<\/code><\/pre>\n<\/div>\n\n\n\n
<\/p>","protected":false},"excerpt":{"rendered":"
Paper Due to the self-similarity of the fingers and severe self-occlusion, it is difficult to predict the correct joint position from local evidence (as shown in Fig. a). By incorporating context information (as shown in Fig. b, where adjacent joints can be accurately predicted), forming enhanced feature maps through pose-guided hierarchical graph (PHG), ambiguity is […]<\/p>\n","protected":false},"author":1,"featured_media":800,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":"","_links_to":"","_links_to_target":""},"categories":[16],"tags":[],"class_list":["post-799","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-depth-based-3d-hand-pose-estimation"],"acf":{"writer":{"simple_value_formatted":"This data type is not supported! Please contact the author for help.<\/em><\/code>","value_formatted":[{"writer_link":{"simple_value_formatted":"Pengfei Ren<\/a>","value_formatted":{"title":"Pengfei Ren","url":"https:\/\/pengfeiren96.github.io\/","target":"_blank"},"value":{"title":"Pengfei Ren","url":"https:\/\/pengfeiren96.github.io\/","target":"_blank"},"field":{"ID":2368,"key":"field_687f0c15f3394","label":"\u4f5c\u8005\u4e0e\u4f5c\u8005\u4e3b\u9875","name":"writer_link","aria-label":"","prefix":"acf","type":"link","value":null,"menu_order":0,"instructions":"","required":1,"id":"","class":"","conditional_logic":0,"parent":2366,"wrapper":{"width":"","class":"","id":""},"return_format":"array","allow_in_bindings":1,"_name":"writer_link","_valid":1,"parent_repeater":"field_687f08dfb7e07"}}},{"writer_link":{"simple_value_formatted":"Haifeng Sun<\/a>","value_formatted":{"title":"Haifeng Sun","url":"https:\/\/scholar.google.com\/citations?user=dwhbTsEAAAAJ&hl=en","target":"_blank"},"value":{"title":"Haifeng Sun","url":"https:\/\/scholar.google.com\/citations?user=dwhbTsEAAAAJ&hl=en","target":"_blank"},"field":{"ID":2368,"key":"field_687f0c15f3394","label":"\u4f5c\u8005\u4e0e\u4f5c\u8005\u4e3b\u9875","name":"writer_link","aria-label":"","prefix":"acf","type":"link","value":null,"menu_order":0,"instructions":"","required":1,"id":"","class":"","conditional_logic":0,"parent":2366,"wrapper":{"width":"","class":"","id":""},"return_format":"array","allow_in_bindings":1,"_name":"writer_link","_valid":1,"parent_repeater":"field_687f08dfb7e07"}}},{"writer_link":{"simple_value_formatted":"Jichang Hao<\/a>","value_formatted":{"title":"Jichang Hao","url":"https:\/\/orcid.org\/0000-0001-6842-4721","target":"_blank"},"value":{"title":"Jichang Hao","url":"https:\/\/orcid.org\/0000-0001-6842-4721","target":"_blank"},"field":{"ID":2368,"key":"field_687f0c15f3394","label":"\u4f5c\u8005\u4e0e\u4f5c\u8005\u4e3b\u9875","name":"writer_link","aria-label":"","prefix":"acf","type":"link","value":null,"menu_order":0,"instructions":"","required":1,"id":"","class":"","conditional_logic":0,"parent":2366,"wrapper":{"width":"","class":"","id":""},"return_format":"array","allow_in_bindings":1,"_name":"writer_link","_valid":1,"parent_repeater":"field_687f08dfb7e07"}}},{"writer_link":{"simple_value_formatted":"Qi Qi<\/a>","value_formatted":{"title":"Qi Qi","url":"https:\/\/scholar.google.com\/citations?user=2W2h0SwAAAAJ","target":"_blank"},"value":{"title":"Qi Qi","url":"https:\/\/scholar.google.com\/citations?user=2W2h0SwAAAAJ","target":"_blank"},"field":{"ID":2368,"key":"field_687f0c15f3394","label":"\u4f5c\u8005\u4e0e\u4f5c\u8005\u4e3b\u9875","name":"writer_link","aria-label":"","prefix":"acf","type":"link","value":null,"menu_order":0,"instructions":"","required":1,"id":"","class":"","conditional_logic":0,"parent":2366,"wrapper":{"width":"","class":"","id":""},"return_format":"array","allow_in_bindings":1,"_name":"writer_link","_valid":1,"parent_repeater":"field_687f08dfb7e07"}}},{"writer_link":{"simple_value_formatted":"Jingyu Wang<\/a>","value_formatted":{"title":"Jingyu Wang","url":"https:\/\/scholar.google.com\/citations?user=H441DjwAAAAJ","target":"_blank"},"value":{"title":"Jingyu Wang","url":"https:\/\/scholar.google.com\/citations?user=H441DjwAAAAJ","target":"_blank"},"field":{"ID":2368,"key":"field_687f0c15f3394","label":"\u4f5c\u8005\u4e0e\u4f5c\u8005\u4e3b\u9875","name":"writer_link","aria-label":"","prefix":"acf","type":"link","value":null,"menu_order":0,"instructions":"","required":1,"id":"","class":"","conditional_logic":0,"parent":2366,"wrapper":{"width":"","class":"","id":""},"return_format":"array","allow_in_bindings":1,"_name":"writer_link","_valid":1,"parent_repeater":"field_687f08dfb7e07"}}},{"writer_link":{"simple_value_formatted":"Jianxin Liao<\/a>","value_formatted":{"title":"Jianxin Liao","url":"https:\/\/dblp.org\/pid\/60\/4951.html","target":"_blank"},"value":{"title":"Jianxin Liao","url":"https:\/\/dblp.org\/pid\/60\/4951.html","target":"_blank"},"field":{"ID":2368,"key":"field_687f0c15f3394","label":"\u4f5c\u8005\u4e0e\u4f5c\u8005\u4e3b\u9875","name":"writer_link","aria-label":"","prefix":"acf","type":"link","value":null,"menu_order":0,"instructions":"","required":1,"id":"","class":"","conditional_logic":0,"parent":2366,"wrapper":{"width":"","class":"","id":""},"return_format":"array","allow_in_bindings":1,"_name":"writer_link","_valid":1,"parent_repeater":"field_687f08dfb7e07"}}}],"value":[{"field_687f0c15f3394":{"title":"Pengfei Ren","url":"https:\/\/pengfeiren96.github.io\/","target":"_blank"}},{"field_687f0c15f3394":{"title":"Haifeng Sun","url":"https:\/\/scholar.google.com\/citations?user=dwhbTsEAAAAJ&hl=en","target":"_blank"}},{"field_687f0c15f3394":{"title":"Jichang Hao","url":"https:\/\/orcid.org\/0000-0001-6842-4721","target":"_blank"}},{"field_687f0c15f3394":{"title":"Qi Qi","url":"https:\/\/scholar.google.com\/citations?user=2W2h0SwAAAAJ","target":"_blank"}},{"field_687f0c15f3394":{"title":"Jingyu Wang","url":"https:\/\/scholar.google.com\/citations?user=H441DjwAAAAJ","target":"_blank"}},{"field_687f0c15f3394":{"title":"Jianxin Liao","url":"https:\/\/dblp.org\/pid\/60\/4951.html","target":"_blank"}}],"field":{"ID":2366,"key":"field_687f08dfb7e07","label":"\u4f5c\u8005","name":"writer","aria-label":"","prefix":"acf","type":"repeater","value":null,"menu_order":0,"instructions":"","required":1,"id":"","class":"","conditional_logic":0,"parent":52,"wrapper":{"width":"","class":"","id":""},"acfe_repeater_stylised_button":0,"layout":"row","pagination":0,"min":0,"max":0,"collapsed":"","button_label":"Add Row","rows_per_page":20,"_name":"writer","_valid":1,"sub_fields":[{"ID":2368,"key":"field_687f0c15f3394","label":"\u4f5c\u8005\u4e0e\u4f5c\u8005\u4e3b\u9875","name":"writer_link","aria-label":"","prefix":"acf","type":"link","value":null,"menu_order":0,"instructions":"","required":1,"id":"","class":"","conditional_logic":0,"parent":2366,"wrapper":{"width":"","class":"","id":""},"return_format":"array","allow_in_bindings":1,"_name":"writer_link","_valid":1,"parent_repeater":"field_687f08dfb7e07"}]}},"\u4f1a\u8bae\u540d\u79f0":{"simple_value_formatted":"TCYB","value_formatted":"TCYB","value":"TCYB","field":{"ID":53,"key":"field_6759c5b33fdb3","label":"\u4f1a\u8bae\u540d\u79f0","name":"\u4f1a\u8bae\u540d\u79f0","aria-label":"","prefix":"acf","type":"text","value":null,"menu_order":1,"instructions":"","required":1,"id":"","class":"","conditional_logic":0,"parent":52,"wrapper":{"width":"","class":"","id":""},"default_value":"\u586b\u5199\u4f1a\u8bae","maxlength":"","allow_in_bindings":1,"placeholder":"","prepend":"","append":"","_name":"\u4f1a\u8bae\u540d\u79f0","_valid":1}},"\u5e74":{"simple_value_formatted":"2021","value_formatted":"2021","value":"2021","field":{"ID":254,"key":"field_675b036a7706e","label":"\u5e74","name":"\u5e74","aria-label":"","prefix":"acf","type":"text","value":null,"menu_order":2,"instructions":"","required":1,"id":"","class":"","conditional_logic":0,"parent":52,"wrapper":{"width":"","class":"","id":""},"default_value":2024,"maxlength":"","allow_in_bindings":1,"placeholder":"","prepend":"","append":"","_name":"\u5e74","_valid":1}},"code":{"simple_value_formatted":"","value_formatted":"","value":"","field":{"ID":54,"key":"field_6759c5dc3fdb4","label":"code","name":"code","aria-label":"","prefix":"acf","type":"link","value":null,"menu_order":3,"instructions":"","required":0,"id":"","class":"","conditional_logic":0,"parent":52,"wrapper":{"width":"","class":"","id":""},"return_format":"array","allow_in_bindings":1,"_name":"code","_valid":1}},"arxiv":{"simple_value_formatted":"","value_formatted":"","value":"","field":{"ID":55,"key":"field_6759c5f83fdb5","label":"arXiv","name":"arxiv","aria-label":"","prefix":"acf","type":"link","value":null,"menu_order":4,"instructions":"","required":0,"id":"","class":"","conditional_logic":0,"parent":52,"wrapper":{"width":"","class":"","id":""},"return_format":"array","allow_in_bindings":0,"_name":"arxiv","_valid":1}},"pdf":{"simple_value_formatted":"PDF<\/a>","value_formatted":{"title":"PDF","url":"https:\/\/ieeexplore.ieee.org\/abstract\/document\/9512523","target":"_blank"},"value":{"title":"PDF","url":"https:\/\/ieeexplore.ieee.org\/abstract\/document\/9512523","target":"_blank"},"field":{"ID":56,"key":"field_6759c6b83fdb6","label":"pdf","name":"pdf","aria-label":"","prefix":"acf","type":"link","value":null,"menu_order":5,"instructions":"","required":0,"id":"","class":"","conditional_logic":0,"parent":52,"wrapper":{"width":"","class":"","id":""},"return_format":"array","allow_in_bindings":0,"_name":"pdf","_valid":1}},"rank":{"simple_value_formatted":"\u4e2d\u79d1\u9662\u4e00\u533a, IF=19.118, \u63a7\u5236\u8bba\u4e0e\u4eba\u5de5\u667a\u80fd\u9886\u57df\u9876\u520a","value_formatted":"\u4e2d\u79d1\u9662\u4e00\u533a, IF=19.118, \u63a7\u5236\u8bba\u4e0e\u4eba\u5de5\u667a\u80fd\u9886\u57df\u9876\u520a","value":"\u4e2d\u79d1\u9662\u4e00\u533a, IF=19.118, \u63a7\u5236\u8bba\u4e0e\u4eba\u5de5\u667a\u80fd\u9886\u57df\u9876\u520a","field":{"ID":2316,"key":"field_686b28a2069eb","label":"\u4f1a\u8bae\/\u671f\u520a\u7ea7\u522b","name":"rank","aria-label":"","prefix":"acf","type":"text","value":null,"menu_order":6,"instructions":"","required":0,"id":"","class":"","conditional_logic":0,"parent":52,"wrapper":{"width":"","class":"","id":""},"default_value":"CCF-A","maxlength":"","allow_in_bindings":0,"placeholder":"","prepend":"","append":"","_name":"rank","_valid":1}}},"_links":{"self":[{"href":"https:\/\/cv.nirc.top\/zh\/wp-json\/wp\/v2\/posts\/799","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cv.nirc.top\/zh\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cv.nirc.top\/zh\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cv.nirc.top\/zh\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/cv.nirc.top\/zh\/wp-json\/wp\/v2\/comments?post=799"}],"version-history":[{"count":47,"href":"https:\/\/cv.nirc.top\/zh\/wp-json\/wp\/v2\/posts\/799\/revisions"}],"predecessor-version":[{"id":2541,"href":"https:\/\/cv.nirc.top\/zh\/wp-json\/wp\/v2\/posts\/799\/revisions\/2541"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cv.nirc.top\/zh\/wp-json\/wp\/v2\/media\/800"}],"wp:attachment":[{"href":"https:\/\/cv.nirc.top\/zh\/wp-json\/wp\/v2\/media?parent=799"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cv.nirc.top\/zh\/wp-json\/wp\/v2\/categories?post=799"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cv.nirc.top\/zh\/wp-json\/wp\/v2\/tags?post=799"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}