Hot Pixel Prediction and Novel Intra Prediction Algorithms for H265 and AV1 Encoders

Download or read book Hot Pixel Prediction and Novel Intra Prediction Algorithms for H265 and AV1 Encoders written by Ariel Shleifer and published by . This book was released on 2018 with total page 136 pages. Available in PDF, EPUB and Kindle. Book excerpt: The demand for streaming video content is on the rise and growing exponentially. Networks bandwidth is very costly and therefore there is a constant effort to improve video compression rates and enable the sending of reduced data volumes while retaining quality of experience (QoE). One basic feature that utilizes the spatial correlation of pixels for video compression is Intra-Prediction, which determines the codec's compression efficiency. Intra prediction enables significant reduction of the Intra-Frame (I frame/key frame) size and, therefore, contributes to efficient exploitation of bandwidth. Therefore, there is high motivation to improve the Intra-Prediction scheme with new, original and proprietary algorithms that will enhance the overall performance of future codecs. In this work, we propose new Intra-Prediction algorithms that improve the AV1 prediction model and provide better compression ratios. Two types of methods are considered: (1) New non-angular Intra-Prediction modes . (2) New scanning order method that maximizes spatial correlation in order to reduce prediction error. Using our initial study of the "hot pixel" problem we've found a concept that can be used for Intra prediction to achieve a better block prediction. The initial study was concluded in our first paper:(1) A. Shleifer, and O. Hadar, "Improvements for hot pixels in digital imagers using lossless approximation techniques," in SPIE Optics + Photonics conference, Proc. of SPIE Vol. 9599, pp. 1-7, 9-13 August 2015, San Diego, California (USA)[7]. In this work, instead of using different angles for predictions, we introduce several unconventional Intra-Prediction modes - Weighted CALIC (WCALIC), Intra-Prediction using System of Linear Equations (ISLE), Prediction of Discrete Cosine Transformations (PrDCT) Coefficients and Reverse Least Power of Three (RLPT) etc. Employed on a selection of eleven typical images with a variety of spatial characteristics, by using Mean Square Error (MSE) evaluation criteria, we show that our proposed algorithms (modes) were preferred and thus selected around 57% of the blocks, resulting in a reduced average prediction error, i.e. the MSE of 26%. These results were published in the second paper:(2) A. Shleifer, C. Lanka, M. Setia, S. Agarwal, O. Hadar and D. Mukherjee, "Novel Intra Prediction modes for VP10 Codec," in SPIE . Optics + Photonics conference, 28th August -1st September 2016, San Diego, California (USA)[12]. Using the current VVC code (JEM) I implemented modes and achieved valuable rate reduction with WCALIC average results: bd-rate -2.4622 ,bd-psnr 0.1915 and ISLE average results: bd-rate -5.9154 ,bd-psnr 0.3356 . Modern video coding standards, including AV1 and HEVC codecs, utilize fixed scan orders in processing blocks during intra coding. The fixed scan orders typically result in residual blocks with high prediction error mainly in blocks with edges. This means that the fixed scan orders cannot fully exploit the content-adaptive spatial correlations between adjacent blocks, thus the bitrate after compression tends to be large. To reduce the bitrate induced by inaccurate intra prediction, the proposed approach adaptively chooses the scanning order of blocks according to criteria of firstly predicting blocks with maximum number of surrounding, already Inter-Predicted blocks. Using the modified scanning method and the new modes has reduced the MSE by up to five times when compared to conventional TM mode with a Raster scan and up to two times when compared to conventional CALIC mode with a Raster scan, depending on the image characteristics (which determines the percentage of blocks predicted with Inter-Prediction, which in turn impacts the efficiency of the new scanning method). These results were published in the paper :(3) A. Shleifer, D. Mukherjee, U. Joshi, I. Mazar, M. Yuzvinsky, N. Tavor, N. Itzhak R. Birman and O. Hadar, "Novel Modes and Adaptive Block Scanning Order for Intra Prediction in AV1,"in SPIE Optics + Photonics conference, 6th - 10th August 2017, 10 pages, San Diego, California (USA). [13]. -- abstract.