基于全局光流的视频插帧算法
首发时间:2021-04-20
摘要:对低帧率视频进行高质量的插帧是计算机视觉的经典问题,然而帧间运动的多样性和复杂性给视频插帧应用带来了极大的挑战。当视频中存在复杂非均匀运动时,根据局部帧间光流估计进行视频插帧可能造成较大偏差。针对此问题,本文提出了一种引入全局光流的视频插帧网络模型Global Optical Flows Interpolation Network (GOFIN)。首先,在中间帧光流预测阶段,设计了一个全局光流预测模块,并探索了两种全局光流的优化方法:使用多项式拟合的光流优化方法和基于卷积LSTM的光流序列优化方法。其次,在插帧生成阶段,设计了一个基于残差增强的合成优化模块,增强特征提取并利用上下文信息进一步消除复杂运动对插帧的影响,提高插帧质量。在UCF-101视频序列数据集上对比多种主流插帧方法的实验表明,全局光流信息的引入对提高视频插帧准确度有极大的意义,本文提出的GOFIN将插帧结果的峰值信噪比提高了0.05db,结构相似性提高了0.069。
For information in English, please click here
Video Frame Interpolation Based on Global Optical Flow
Abstract:High-quality interpolation of low frame rate video is a classic problem of computer vision. However, the diversity and complexity of motion between frames have brought great challenges to the application of video interpolation. When there is complex and non-uniform motion in the video, video interpolation based on local inter-frame optical flow estimation may cause large deviations. In response to the problem, the paper proposes a Global Optical Flows Interpolation Network (GOFIN), a video interpolating network model that introduces global optical flow. First, in the intermediate frame optical flow prediction stage, a global optical flow prediction module was designed, and two optimization methods for global optical flow were explored: optical flow optimization method using polynomial fitting and optical flow sequence optimization based on convolution LSTM method. Secondly, in the generation stage of the inserted frame, a synthesis optimization module based on residual enhancement is designed to enhance feature extraction and use context information to further eliminate the influence of complex motion on the inserted frame and improve the quality of the inserted frame. Experiments comparing a variety of mainstream frame interpolation methods on the UCF-101 video sequence data set show that the introduction of global optical flow information is of great significance to improve the accuracy of video frame interpolation. The GOFIN proposed in this paper reduces the peak value of the frame interpolation results. The noise ratio is improved by 0.05db, and the structural similarity is improved by 0.069.
Keywords: video frame interpolation global optical flow residual learning
基金:
引用
No.****
动态公开评议
共计0人参与
勘误表
基于全局光流的视频插帧算法
评论
全部评论0/1000