Joint Hand Detection and Rotation Estimation Using CNN

Xiaoming Deng¹ Yinda Zhang² Shuo Yang¹ Ping Tan³ Liang Chang⁴ Ye Yuan¹ Hongan Wang¹

¹ Institute of Software CAS ² Princeton University ³ Simon Fraser University ⁴ Beijing Normal University

Abstract

Hand detection is essential for many hand related tasks, e.g. parsing hand pose, understanding gesture, which are extremely useful for robotics and human-computer interaction. However, hand detection in uncontrolled environments is challenging due to the flexibility of wrist joint and cluttered background. We propose a deep learning based approach which detects hands and calibrates in-plane rotation under supervision at the same time. To guarantee the recall, we propose a context aware proposal generation algorithm which significantly outperforms the selective search. We then design a convolutional neural network(CNN) which handles object rotation explicitly to jointly solve the object detection and rotation estimation tasks. Experiments show that our method achieves better results than state-of-the-art detection models on widely-used benchmarks such as Oxford and Egohands database. We further show that rotation estimation and classification can mutually benefit each other.

Video

Paper

X. Deng, Y. Zhang, S. Yang, P. Tan, L. Chang, Y. Yuan, H. Wang.
"Joint Hand Detection and Rotation Estimation Using CNN"
IEEE Transactions on Image Processing, 27(4):1888-1900.

Download

The rotation annotation of hand detection dataset will be available.