Streaming Video over the Internet Streaming Video over the Internet Approaches and DirectionsApproaches and Directions

Dapeng WuYiwei Thomas HouWenwu ZhuYa-Qin ZhangJon M. Peha

2001 년 9 월 4 일

발표자 : 서승우

Raw Video

CompressedAudio

Application-layerQoS ControlCompressed

Video

TransportProtocol

VideoDecoder

AudioDecoder

Raw Audio

VideoCompression

AudioCompression

Application-layerQoS Control

TransportProtocol

MediaSynchronization

Storage Device

Streaming Server

Client/Receiver

Internet(Continuous media distribution services)

Six Key AreasSix Key Areas

Video Compression Application layer QoS control Continuous media distribution services Streaming servers Media Synchronization mechanisms Protocols for Steaming media

Two approaches of video compressionTwo approaches of video compression

Nonscalable video coding– Generate 1 compressed bit-stream

Scalable video coding– Generate multiple compressed bit-stream– One is base substream– Others are enhancement substream– SNR , FGS

Six Key AreasSix Key Areas

Video CompressionApplication layer QoS control Continuous media distribution services Streaming servers Media Synchronization mechanisms Protocols for Steaming media

Application-Layer QoS ControlApplication-Layer QoS Control

Congestion Control– Rate Control

Source-Based Rate Control

Receiver-Based Rate Control

Hybrid-Rate Control– Rate Shaping

Frame-dropping filter Layer-dropping filter Frequency filter Re quantization filter

Error Control– FEC– Delay-Constrained

Retransmission– Error-Resilient

Encoding– Error-Concealment

To avoid congestion and maximize video quality in the presence of packet loss

Rate ControlRate Control

Source-Base Rate Control– Probe-Based

Maintain their packet loss ratio below a certain threshold

– Model-Based Based on throughput model

Rate ControlRate Control

Receiver-Based Rate-Control– When no congestion is detected,join a

layer/channel =>Result in an increase of its receiving rate

– When congestion detected, drops a layer

=> result in a reduction of its receiving rate Hybrid Rate-Control

– Both Sender and receiver control rate

Rate ShapingRate Shaping

To match the rate of a pre-compressed video bit stream to their target rate constraint

Required for source based rate control

Codec filter Frame-dropping filter Layer-dropping filter Frequency filter Re-quantization filter

Error ControlError ControlFEC: add redundant informationFEC: add redundant information

Channel coding :– Packetized into k packets– Generate n packet block ( k < n)– Only need to receive any k packets in n packet block

Source coding-based FEC– N-th group of blocks contains redundant information

of (n-1)th group of blocks Joint source/channel coding

– Optimal allocation between source coding and channel coding

Error ControlError ControlDelay-Constrained RetransmissionDelay-Constrained Retransmission

Error ControlError ControlError-Resilient EncodingError-Resilient Encoding

MDC : Multiple description coding– Compressed into multiple streams– Each description provides acceptable

visual quality– With only one description, can

reconstruct video with acceptable quality

– COST !!

Error ControlError ControlError ConcealmentError Concealment

Reconstruct the missing data in intra-coded frames in inter-coded frames

Example of skills– Maximally smooth recovery– projection onto convex sets– various motion vector– coding mode recovery methods

Six Key AreasSix Key Areas

Video Compression Application layer QoS controlContinuous media distribution service

s Streaming servers Media Synchronization mechanisms Protocols for Steaming media

Continuous Media Distribution ServicesContinuous Media Distribution Services

Network FilteringApplication-Level MulticastContent Replication

Network FilteringNetwork Filtering

Network FilteringNetwork Filtering

Typically, frame-dropping filters are used as network filters

Application-Level MulticastApplication-Level Multicast

IP multicast has barrier– Scalability,network

management,deployment,support for higher layer functionality

=> application level multicast mechanism was proposed

Media-Bridge– Performs routing at application layer– Interconnect with neighbor Media-Brides through

explicit configuration– Application level multicast is possible

Content ReplicationContent Replication

Mirroring– Place copies of the original multimedia files

on other machines scattered around the Internet

Caching– Makes local copies of contents that clients

retrieve frequently

– To store complete video is expensive– A few cached frames can contribute to

significant improvement in performance

Six Key AreasSix Key Areas

Video Compression Application layer QoS control Continuous media distribution servicesStreaming servers Media Synchronization mechanisms Protocols for Steaming media

Streaming ServersStreaming Servers

Real-Time Operation SystemStorage System

Streaming ServersStreaming ServersOS:process managementOS:process management

EDF : each task is assigned deadline – close to deadline first

Rate-monotonic scheduling– high rate task first– More task switching

< EDF vs Rate monotonic >

Streaming ServersStreaming ServersOS:Resource ManagementOS:Resource Management

Resource management involves admission control and resource allocation

Deterministic admission control– Hard guarantees

Statistical admission control– Statistical guarantees

Streaming ServersStreaming ServersOS:File ManagementOS:File Management

Non-scatterd vs scattedScatted

– throughput improved– Seek times can be reduced by disk-schedul

ing algorithms SCAN-EDF, grouped sweeping scheduling,DC-SCAN

Support pause/resume,fast forward,fast backward => under study

Streaming ServersStreaming ServersStorage SystemStorage System

Increase throughput with data striping– load-balance and low latency are two conflicting objectives

Streaming ServersStreaming ServersStorage SystemStorage System

Increase capacity with tertiary and hierarchical storage– To keep the storage cost down , tertiary

storage must be added

SAN : use fiber channel

NAS : directly attach to a local area network

Six Key AreasSix Key Areas

Video Compression Application layer QoS control Continuous media distribution services Streaming serversMedia Synchronization mechanisms Protocols for Steaming media

Media SynchronizationMedia Synchronization

Intra-stream synchronization– Maintain continuity of logical data

unitInter-stream synchronization

– Maintain temporal relationships among different continuous media

Inter-object synchronization– Relation with time-independent data

such as text, still image and streams

Media SynchronizationMedia Synchronization

Methods maintain the temporal relations– Interval-based– Axes-based : time stamping

Widely used

– Control flow-based– Event-based

Six Key AreasSix Key Areas

Video Compression Application layer QoS control Continuous media distribution services Streaming servers Media Synchronization mechanismsProtocols for Steaming media

Protocols For Streaming VideoProtocols For Streaming Video

Transport ProtocolsSession Control Protocols

Protocols For Steaming VideoProtocols For Steaming VideoTransport ProtocolsTransport Protocols

TCP – retransmission

UDP– Typically employed as the transport

protocol

Protocols For Steaming VideoProtocols For Steaming VideoTransport ProtocolsTransport Protocols

RTP : does not guarantee QoS or reliable delivery

– Time stamping– Sequence numbering– Payload type identificatio

n– Source identification

RTCP : designed to work in conjunction with RTP

– QoS feedback– Participant identification– Control packets scaling– Inter-media synchronizati

on– Minimal session control i

nformation

Protocols For Steaming VideoProtocols For Steaming VideoSession Control ProtocolsSession Control Protocols

RTSP– Support VCR-like control operations– (stop,pause/resume,fast forward, fast backward)– Choosing delivery channel(UDT,multicast UDT, TCP)– Establish and control streams of continuous audio and video

media– Media retrieval : media request,ask session setup to server– Adding media to an existing session

SIP– Support user mobility by proxying and redirecting request