In-air Handwriting Data Acquisition System Based on Monocular Vision

Aiming at the shortcomings of the large and expensive three-dimensional (3D) sensor used in a vision-based in-air handwriting system, we propose a new type of in-air handwriting data acquisition system based on monocular vision. The system uses a small and cheap monocular camera to replace the 3D sensor such as Kinect and Leap Motion, which is more suitable for integration and large-scale promotion with household devices such as mobile phones, air conditioners, and televisions. To solve the problem that existing in-air handwriting systems based on monocular vision can only write a single character or short text each time, and the length of the user's written text is limited, we propose a coordinate system virtual sliding technology. First, the system uses the background difference method and the HSV color space shadow elimination method to obtain the complete hand contour from the two-dimensional video frame. Second, we separate the palm and finger using the structural characteristics of the hand. Using the constructed fingertip template, we determine the position of the fingertip by matching the segmented fingers. Finally, we connect the fingertip positions in continuous video frames to form in-air handwritten character trajectories. Through the proposed coordinate system virtual sliding technology, users can write continuously to form texts. The experimental results show that the system can allow users to write continuously in the air freely, and the detection accuracy of fingertips reaches 96.8%.