Robust Electronic Image Stabilization Method Based on IMU-Camera Calibration

FANG Ming; TIAN Ying

doi:10.13976/j.cnki.xk.2018.0156

Volume 47 Issue 2

Apr. 2018

Turn off MathJax

Article Contents

Abstract

References

INFORMATION AND CONTROL > 2018 > 47(2): 156-165. > DOI: 10.13976/j.cnki.xk.2018.0156 CSTR: 32166.14.xk.2018.0156

FANG Ming, TIAN Ying. Robust Electronic Image Stabilization Method Based on IMU-Camera Calibration[J]. INFORMATION AND CONTROL, 2018, 47(2): 156-165. DOI: 10.13976/j.cnki.xk.2018.0156

Citation:

FANG Ming, TIAN Ying. Robust Electronic Image Stabilization Method Based on IMU-Camera Calibration[J]. INFORMATION AND CONTROL, 2018, 47(2): 156-165. DOI: 10.13976/j.cnki.xk.2018.0156

Citation:

FANG Ming, TIAN Ying. Robust Electronic Image Stabilization Method Based on IMU-Camera Calibration[J]. INFORMATION AND CONTROL, 2018, 47(2): 156-165. DOI: 10.13976/j.cnki.xk.2018.0156

PDF (1144 KB)

Robust Electronic Image Stabilization Method Based on IMU-Camera Calibration

FANG Ming^,,
TIAN Ying

School of Computer Science and Technology, Changchun University of Science and Technology, Changchun 130022, China

More Information

Received Date: November 27, 2016
Revised Date: March 06, 2017
Accepted Date: February 16, 2017
Available Online: December 01, 2022
Published Date: April 19, 2018

Graphical Abstract

Abstract

Abstract

We propose an electronic image stabilization method to attach an inertial measurement unit (IMU) to a camera. Then, we apply the IMU data to estimate the attitude changes of the camera indirectly and in real time to implement feedback transformation on real-time images. First, we calibrate the IMU and the camera coordinate system by using the quaternion method to conjoin the two coordinate systems. Then, we adopt the least square method to complete the minimum error match of the calibrated IMU and the camera to achieve trigger time synchronization. Finally, we use the IMU's attitude information to feedback the camera's jitter in real time to realize video stabilization. Experimental results show that the effects of IMU stabilization and scale-invariant feature transform (SIFT) are almost the same in a normal scene. In addition, in a scarce texture information scene, the stabilization correlation of the IMU increases by approximately 20% in comparison with the SIFT method and by 40% in comparison with the gray projection method. This method breaks through the limitations of application scenarios, and has high robustness and application value.
- electronic image stabilization,
- inertial measurement unit (IMU),
- quaternion,
- time synchronization,
- calibration,
- jitter

FullText(HTML)

References (20)

References

[1]	李计添, 何永强, 陈财森, 等.一种基于灰度投影算法的车载电子稳像方法[J].红外技术, 2010, 32(6):328-332. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=hwjs201006005 Li J T, He Y Q, Chen C S, et al. A method of vehicle-bone electronic image stabilization based on gray projection algorithm[J]. Infrared Technology, 2010, 32(6):328-332. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=hwjs201006005
[2]	Vella F, Castorina A, Mancuso M, et al. Digital image stabilization by adaptive block motion vectors filtering[J]. IEEE Transactions on Consumer Electronics, 2002, 48(3):796-801. doi: 10.1109/TCE.2002.1037077
[3]	Zhang Y, Yao H, Xu P, et al. Video stabilization based on saliency driven SIFT matching and discriminative RANSAC[C]//the Third International Conference on Internet Multimedia Computing and Service. New York, NJ, USA: ACM, 2011: 65-69.
[4]	Cheng L, Wang X P, Meng L Y, et al. Electronic image stabilization for railway crossing video based on gray projection[J]. Computer Engineering & Design, 2009, 30(2):443-445. http://en.cnki.com.cn/Article_en/CJFDTOTAL-SJSJ200902054.htm
[5]	Cai J H, Walker R. Robust video stabilisation algorithm using feature point selection and delta optical flow[J]. IET Computer Vision, 2009, 3(4):176-188. doi: 10.1049/iet-cvi.2009.0036
[6]	张永祥. 基于陀螺仪的船载电子稳像技术研究[D]. 长春: 长春光学精密机械与物理研究所, 2006. Zhang Y X. Study on electronic image stabilization technology for the image sequences of the ship-borne camera system base on the gyro[D]. Changchun: Changchun Institute of Optic, Fine Mechanics and Physics, Chinese Academy of Sciences, 2006.
[7]	王日俊. 多旋翼无人飞行器载荷稳像技术研究[D]. 长春: 长春光学精密机械与物理研究所, 2015. Wang R J, Study on image stabilization technology for the payload of mUAV[D]. Changchun: Changchun Institute of Optic, Fine Mechanics and Physics, Chinese Academy of Sciences, 2015.
[8]	马增辉, 刘长良.一类非最小相位系统的PID控制器整定方法[J].信息与控制, 2015, 44(2):147-151. http://ic.sia.cn/CN/abstract/abstract12259.shtml Ma Z H, Liu C L. Tuning method for a PID controller in a non-minimum phase system[J]. Information and Control, 2015, 44(2):147-151. http://ic.sia.cn/CN/abstract/abstract12259.shtml
[9]	Horn B K P, Hilden H M, Negahdaripour S. Closed-form solution of absolute orientation using orthonormal matrices[J]. Journal of the Optical Society of America A, 1988, 5(7):1127-1135. doi: 10.1364/JOSAA.5.001127
[10]	邱茂林, 马颂德, 李毅.计算机视觉中摄像机定标综述[J].自动化学报, 2000, 26(1):43-55. https://www.wenkuxiazai.com/doc/f60d1ad97f1922791688e812-2.html Qiu M L, Ma S D, Li Y. Overview of camera calibration for computer vision[J]. Acta Automatica Sinica, 2000, 26(1):43-55. https://www.wenkuxiazai.com/doc/f60d1ad97f1922791688e812-2.html
[11]	Zhang Z. A flexible new technique for camera calibration[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2000, 22(11):1330-1334.
[12]	吴哲明, 孙振国, 张文增, 等.基于惯性测量单元旋转的陀螺漂移估计和补偿方法[J].清华大学学报:自然科学版, 2014, 54(9):1143-1147. http://industry.wanfangdata.com.cn/dl/Detail/Patent?id=Patent_CN201310750475.8&type=Free Wu Z M, Sun Z G, Zhang W Z, et al. Gyroscope bias estimation and compensation by rotation of the inertial measurement unit[J]. Journal of Tsinghua University:Science and Technology, 2014, 54(9):1143-1147. http://industry.wanfangdata.com.cn/dl/Detail/Patent?id=Patent_CN201310750475.8&type=Free
[13]	Horn B K. Closed-form solution of absolute orientation using unit quaternions[J]. Journal of the Optical Society of America A, 1987, 4(4):629-642. doi: 10.1364/JOSAA.4.000629
[14]	Tian Y, Fang M. A time synchronization method for inertial sensor and visual sensor[C]//12th World Congress on Intelligent Control and Automation. Piscataway, NJ, USA: IEEE, 2016: 836-839.
[15]	孙茂伟, 杨慧中.局部加权混合核偏最小二乘算法及其在软测量中的应用[J].信息与控制, 2015, 44(4):481-486. http://ic.sia.cn/CN/abstract/abstract12311.shtml Sun M W, Yang H Z. Local weighted mixed kernel partial least squares algorithm and its applications in to soft-sensing[J]. Information and Control, 2015, 44(4):481-486. http://ic.sia.cn/CN/abstract/abstract12311.shtml
[16]	Hearn D, Baker M P. Computer graphics:C version[M]. 2nd ed. New Jersey, USA:Prentice-Hall, Inc. 1996:610-662.
[17]	Wang H, Dan O, Verbeek J. A robust and efficient video representation for action recognition[J]. International Journal of Computer Vision, 2015, 36(2):1-20. http://arxiv.org/abs/1504.05524
[18]	Jia C, Sinno Z, Evans B L. Real-time 3D rotation smoothing for video stabilization[C]//Asilomar Conference on Signals, Systems and Computers. Piscataway, NJ, USA: IEEE, 2015: 673-677.
[19]	Chen X, Song R, Shen C, et al. Application of a genetic algorithm Elman network in temperature drift modeling for a fiber-optic gyroscope[J]. Applied Optics, 2014, 53(26):6043-6050. doi: 10.1364/AO.53.006043
[20]	黄晨, 王建军, 高昕, 等.电子稳像中稳像质量评价方法研究[J].激光与红外, 2013, 43(5):477-481. http://www.wanfangdata.com.cn/details/detail.do?_type=degree&id=Y2122148 Huang C, Wang JJ, Gao X, et al. Study on video stabilization performance assessment inelectronic image stabilization[J]. Laser & Infrared, 2013, 43(5):477-481. http://www.wanfangdata.com.cn/details/detail.do?_type=degree&id=Y2122148