This paper presents several error resilient video coding methods based on redundant pictures. We combine redundant picture coding with reference picture selection and reference picture list reordering to prevent error propagation in motion compensated video coding. A hierarchical redundant picture allocation method is employed to make a tradeoff between error resilience and coding efficiency. For improved end-to-end rate-distortion performance in packet loss environment, three adaptive redundant picture allocation methods are further developed, utilizing characteristics of the input video content. Simulation results show that the adaptive redundant picture coding methods can achieve average luma peak-signal-to-noise improvements up to 3.5 dB compared to the loss-aware rate distortion optimized intra macroblock refresh algorithm implemented in the H.264/AVC Joint Model (JM). The proposed redundant picture coding methods are standard-compliant and do not introduce any additional end-to-end delay, therefore suit for low-delay applications such as video telephony and video conferencing. Due to the good error resilience performance, some of the proposed redundant picture coding methods have been adopted and integrated into the JM.

In this paper, a video inpainting approach is proposed, which targets at repairing a video containing moving humans that are largely or completely occluded or missing for some of the frames. The proposed approach first categorizes typically periodic human motion in a video into a set of temporal states (called motion states), and then estimates the motion states for the frames with missing humans so as to repair the missing parts using other undamaged frames with the same motion states. This deviates from common approaches that directly repair the pixels of the damaged parts. Experiments demonstrate that the proposed method can well repair the damaged video sequences without introducing strong artifacts that exist in many existing techniques.

In theory, a Wyner-Ziv video codec should achieve the same efficiency as a joint encoding and decoding one. However, existing approaches still exhibit significant gaps. One main reason is the lack of the complete correlation exploitation between source and side information. In this paper, we propose a transform domain classification based Wyner-Ziv video codec, aiming at exploiting additional video statistics. In this proposed new scheme, the encoder exploits additional statistics by performing block classification to differentiate low motion blocks from high motion ones. In general, low motion blocks represent highly correlated regions. Such information is useful when the decoder performs motion-compensated interpolation to obtain better side information, thus improving the performance of Wyner-Ziv coding. Experimental results show that we are indeed able to achieve better rate-distortion performance compared to the existing Wyner-Ziv video codecs at the expense of some additional complexity in frame store and comparisons at the encoder.

This paper presents a comparison between the scalable extension of H.264/AVC (SVC) and transcoding, in terms of Rate-Distortion(RD) performance and computational complexity. Both bitrate and spatial resolution reduction cases are included. Simulation results show that, in the aspect of RD performance, SVC performs significantly better in bitrate reduction and slightly better in spatial resolution reduction. Meanwhile, SVC adaptation through bitstream extraction has negligible complexity compared with transcoding1.

With the growing popularity of personal digital assistant devices and smart phones, more and more consumers are becoming quite enthusiastic to appreciate videos via mobile devices. However, limited display size of the mobile devices has been imposing significant barriers for users to enjoy browsing high-resolution videos. In this paper, we present an attention-information-based spatial adaptation framework to address this problem. The whole framework includes two major parts: video content generation and video adaptation system. During video compression, the attention information in video sequences will be detected using an attention model and embedded into bitstreams with proposed supplement-enhanced information (SEI) structure. Furthermore, we also develop an innovative scheme to adaptively adjust quantization parameters in order to simultaneously improve the quality of overall encoding and the quality of transcoding the attention areas.When the high-resolution bitstream is transmitted to mobile users, a fast transcoding algorithm we developed earlier will be applied to generate a new bitstream for attention areas in frames. The new low-resolution bitstream containing mostly attention information, instead of the high-resolution one, will be sent to users for display on the mobile devices. Experimental results show that the proposed spatial adaptation scheme is able to improve both subjective and objective video qualities.