Scalable ROI Algorithm for H.264/SVC-Based Video Streaming
description
Transcript of Scalable ROI Algorithm for H.264/SVC-Based Video Streaming
![Page 1: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/1.jpg)
Scalable ROI Algorithm for H.264/SVC-Based Video Streaming
Jung-Hwan Lee and Chuck Yoo, Member, IEEE
![Page 2: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/2.jpg)
Overviews
• Introduction• H.264/SVC• Region of Interests• System Architecture• Experimental Results• Conclusion
![Page 3: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/3.jpg)
Introduction
![Page 4: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/4.jpg)
Introduction
• Why SVC?
• What is ROI?
• How to combine SVC and ROI?
• Using FGS for example
![Page 5: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/5.jpg)
Bandwidth Variation
![Page 6: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/6.jpg)
Packet-losses
![Page 7: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/7.jpg)
ROI
![Page 8: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/8.jpg)
Combine SVC and ROI
Use real-time face-detection algorithms
![Page 9: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/9.jpg)
Combine SVC and ROI (cont.)
![Page 10: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/10.jpg)
Introduction
• The authors propose a scalable ROI algorithm, which can support fine-grained scalability in region of interests with low computing complexity.
![Page 11: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/11.jpg)
H.264/SVC
![Page 12: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/12.jpg)
H.264/SVC
• H.264/SVC made adaptive bit rate control according to network condition and resolution control according to device capability possible.
![Page 13: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/13.jpg)
H.264/SVC Enhancement
• Spatial • Temporal • SNR (Quality)
![Page 14: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/14.jpg)
Spatial Enhancement
Reference : H.264 and MPEG-4 VIDEO COMPRESSION, Iain E.G. Richardson
![Page 15: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/15.jpg)
Temporal Enhancement
Reference : Overview of The Scalable Video Coding Extension, SCHWARZ et al
![Page 16: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/16.jpg)
Quality Enhancement
• The multilayer concept for quality scalable coding allows a few selected bit rates to be supported in a scalable bit stream.
![Page 17: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/17.jpg)
Quality Enhancement (cont.)
Texture FDCT Quant Encode coefficient
s
Rescale
Encode each
bitplane
Baselayer
Enhancementlayer
Figure FGS encoder block diagram (simplified)
![Page 18: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/18.jpg)
ROI • Passive Setting of ROI• Active Setting of ROI
![Page 19: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/19.jpg)
Passive Setting of ROI
• The aim of ROI coding is to set a high resolution in ROI and low resolution in nonROI.
• Methods of setting ROI• Passive setting of ROI
• Define regions of interest beforehand
• Active setting of ROI• Constantly change according to environment or
contents
![Page 20: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/20.jpg)
Active Setting of ROI
![Page 21: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/21.jpg)
System Architecture
• Method of structuring ROI• Scalable ROI Algorithm
![Page 22: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/22.jpg)
System Architecture
![Page 23: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/23.jpg)
System Architecture (cont.)
• Two processes need to be defined beforehand.• H.264/SVC video file is encoded with the SNR
enhancing MGS method.• QoE monitor is needed to regularly check the network
status.
• Through this process, the algorithm controls the enhancement layers and the range of ROIs.
![Page 24: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/24.jpg)
Method of structuring ROI
• Passive method of setting ROI is used in this study.
• Center of the screen is set as ROI and areas far from the screen are non-ROIs.
• FMO Box-Out method is applied and ROIs are divided into three stages (Slice Group) .
![Page 25: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/25.jpg)
Method of structuring ROI (cont.)
• After the steps mentioned, the scalable ROI layers are extracted in three different forms.
![Page 26: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/26.jpg)
Scalable ROI algorithm
• The scalable ROI algorithm is applied to the existing bit stream extractor functions.
Reference : JSVM 9_18 software manual
![Page 27: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/27.jpg)
Scalable ROI algorithm (cont.)
• As shown in Fig. 6, ROI algorithm extracted models needs the elements in Table below.
![Page 28: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/28.jpg)
Scalable ROI algorithm (cont.)
• Bw() must (not) be more than the total sum of the basic layer, the layer without SR (Scalable ROI layer) application and the upper layer with SR application.
1. Basic layer.2. The SNR level
range that is not set as ROI.
1. Upper layers with SR application.
2. Enhanced layers set as ROI.
![Page 29: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/29.jpg)
Scalable ROI algorithm (cont.)
• is the SNR dividing coefficient of selected layers, and has the maximum value of MGS division.
• has a different value according to MGS quality in regions with SR application, calculated with eq(2),(3),(4)
1. Basic layer.2. The SNR level
range that is not set as ROI.
1. Upper layers with SR application.
2. Enhanced layers set as ROI.
![Page 30: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/30.jpg)
Scalable ROI algorithm (cont.)
• (2) try to extract ROI from the overall screen. But ROI method cannot applied because the number of quality flags does not meet the minimum value for which video improvement is possible after extraction.
![Page 31: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/31.jpg)
Scalable ROI algorithm (cont.)
• (3) is the case where the quality flags are applied most to the top layer of the overall screen. Since the layer with the highest quality flag value in the overall screen changes in quality flag number in layers according to the j value, this indicates that the size of the ROI screen changes.
![Page 32: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/32.jpg)
Scalable ROI algorithm (cont.)
• (4) is when the quality flag value is half or more of the overall screen. The SR is applied differently to different screen sizes according to the size of the bandwidth and number of quality flags on the screen.
![Page 33: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/33.jpg)
Experimental Result
![Page 34: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/34.jpg)
Experiment Environment
• The JVSM version 9.13 is used.
![Page 35: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/35.jpg)
Experiment Environment (cont.)
![Page 36: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/36.jpg)
Experiment Environment (cont.)
![Page 37: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/37.jpg)
Experiment Environment
• Figure shows PSNR between comparison between traditional and proposed methods.
• Proposed method confirms ROI areas have higher PSNR than non-ROI areas.
![Page 38: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/38.jpg)
Conclusion
![Page 39: Scalable ROI Algorithm for H.264/SVC-Based Video Streaming](https://reader036.fdocument.pub/reader036/viewer/2022081505/568164ca550346895dd6e645/html5/thumbnails/39.jpg)
Conclusion
• Traditional CGS cannot provide high video quality when the network condition is unstable.
• Proposed method support high subjective quality with FGS by applying ROI to H.264/SVC.