Drawings
Figure 1 illustrates a simplified block diagram of a communication system for providing rate adaptation in adaptive streaming environments in accordance with one embodiment of the present disclosure;
Turning to Figure 1, Figure 1 is a simplified block diagram of a communication system 100 configured for providing rate adaptation for a plurality of HAS client(s) in accordance with one embodiment of the present disclosure. Communication system 100 may include a plurality of servers 102, a media storage 106, a network 108, a transcoder 104, a plurality of HAS client(s) 112, and a plurality of intermediate nodes 110. Note that the originating video source may be a transcoder that takes a single encoded source and “transcodes” it into multiple rates, or it could be a “Primary” encoder that takes an original non-encoded video source and directly produces the multiple rates. Therefore, it should be understood that transcoder 104 is representative of any type of multi-rate encoder, transcoder, etc.
Servers 102 are configured to deliver requested content to HAS client(s) 112-c. The content may include any suitable information and/or data that can propagate in the network (e.g., video, audio, media, any type of streaming information, etc.). certain content may be stored in media storage 106, which can be located anywhere in the network. Media storage 106 may be a part of any web server, logically connected to one of servers 102, suitably accessed using network 108, etc. In general, communication system 100 can be configured to provide downloading and streaming capabilities associated with various data services. Communication system 100 can also offer the ability to manage content for mixed-media offerings, which may combine video, audio, games, applications, channels, and programs into digital media bundles.
In accordance with the techniques of the present disclosure, the architecture of Figure 1 can provide a new rate adaptation framework that includes several significant mechanisms. First, the architecture can use an algorithm to adjust a flow’saverage throughput to match the available bandwidth. Second, the architecture can fine-tune the intervals between consecutive segment downloads. Additionally, the framework can offer an enhancement (via a new time discount addition) to an additive-increase/multiplicative-decrease (AIMD) equation, as detailed below. It can also make use of the fine-tuned interval between consecutive segment downloads, where the rate adaptation achieves weighted bandwidth sharing regardless of the underlying transport protocol’s (e.g., TCP, SCTP, MP-TCP, etc.) Behavior.
In certain example embodiments, the proposed rate adaptation algorithm can effectively mitigate the frequent rate shifts (e.g., rate oscillation) problems commonly experienced by typical HAS client(s), especially when multipleHAS client(s) compete for bandwidth at one or more network bottleneck links. Typical HAS client(s) simply rely on the underlying TCP’sbandwidth sharing behavior to choose a bitrate. By contrast, the framework discussed herein is able to decouple the bitrate selection from its underlying TCP’sbandwidth sharing behavior. Many existing HAS client(s) implement a symmetric rate upshift/downshift. Embodiments of the present disclosure are able to achieve downshifts more responsively than the upshifts.
One significant aspect of example embodiments of the present disclosure includes an algorithm for proactively probing for available network bandwidth by requesting higher-bitrate video segments. In addition, certain embodiments of the present disclosure can apply to both the streaming of stored and live contents. Additionally, in implementing the probe-adapt principle and the fine-tuning of inter-request intervals, certain embodiments of the present disclosure can achieve both high video rates and excellent video rate stability.
Before detailing these activities in more explicit terms, it is important to understand some of the bandwidth challenges encountered in a network that includes HAS client(s). The following foundational information may be viewed as a basis from which the present disclosure may be properly explained. Adaptive streaming video systems make use of multi-rate video encoding and an elastic IP transport protocol suite (typically hypertext transfer protocol/transmission control protocol/Internet protocol (HTTP/TCP/IP), but could include other transports such as HTTP/SPDY/IP, etc.) to deliver high-quality streaming video to a multitude of simultaneous users under widely varying network conditions. These systems are typically employed for “over-the-top” video services, which accommodate varying quality of service over network paths.
In adaptive streaming, the source video is encoded such that the same content is available for streaming at a number of different rates (this can be via either multi-rate coding, such as H.264 AVC, or layered coding, such as H.264 SVC). The video can be divided into “chunks” of one or more group-of-pictures (GOP) (e.g., typically two (2) to ten (10) seconds of length). HAS client(s) can accesschunks stored on servers (or produced in near real-time for live streaming) using a web paradigm (e.g., HTTP GET operations over a TCP/IP transport), and they depend on the reliability, congestion control, and flow control features of TCP/IP for data delivery. HAS client(s) can indirectly observe the performance of the fetch operations by monitoring the delivery rate and/or the fill level of their buffers and, further, either upshift to a higher encoding rate to obtain better quality when bandwidth is available, or downshift in order to avoid buffer underruns and the consequent video stalls when available bandwidth decreases, or stay at the same rate if available bandwidth does not change. Compared to inelastic systems such as classic cable TV or broadcast services, adaptive streaming systems use significantly larger amounts of buffering to absorb the effects of varying bandwidth from the network.
In a typical scenario, HAS client(s) would fetch content from a network server in segments. Each segment can contain a portion of a program, typically comprising a few seconds of program content. [Note that the term `segment` and `chunk` are used interchangeably in this disclosure.] For each portion of the program, there are different segments available with higher and with lower encoding bitrates: segments at the higher encoding rates require more storage and more transmission bandwidth than the segments at the lower encoding rates. HAS client(s) adapt to changing network conditions by selecting higher or lower encoding rates for each segment requested, requesting segments from the higher encoding rates when more networkbandwidth is available (and/or the client buffer is close to full), and requesting segments from the lower encoding rates when less network bandwidth is available (and/or the client buffer is close to empty).
With most adaptive streaming technologies, it is common practice to have every segment represent the same, or very nearly the same, interval of program time. For example, in the case of one streaming protocol, it is common practice to have every segment (referred to as a `fragment`) of a program represent almost exactly 2 seconds worth of content for the program. With HTTP Live Streaming (HLS), it is quite common practice to have every segment of a program represent almost exactly 10 seconds worth of content. Although it is also possible to encode segments with different durations (e.g., using 6-second segments for HLS instead of 10-second segments), even when this is done, it is nevertheless common practice to keep all segments within a program at the same duration.
Figure 2 illustrates a simplified block diagram illustrating a possible adaptive streaming scenario;
Turning to Figure 2, Figure 2 is a simplified block diagram illustrating an environment 200 for providing adaptive videostreaming over HTTP. This particular system can include a media player, a client buffer, multiple service providers, multiple content providers, and a network over which content can be exchanged. A client can download the segments in any order using plain HTTP GETs, measure the available bandwidth based on the download history, and select the video bitrate of the next segment on-the-fly. Typically, tens of seconds of downloaded video segments are buffered at the client to absorb unexpected bandwidth fluctuation. A viable rate adaptation algorithm should generally yield a high average video quality, a low variation of video quality, and offer a low chance of video playout stall caused by buffer underruns.
Certain embodiments of the present disclosure can provide a new rate adaptation algorithm for adaptive streaming that achieves several potential benefits. First, typical HAS client(s) estimate the available bandwidth by equating it to the measured throughput of downloading the previous several segments (i.e., historical data, which is inclusive of any information associated with previous media activity). When two or more HAS client(s) compete for bandwidth at some network bottleneck link, this turns out to be an inappropriate way of estimating bandwidth and, further, doing so results in frequent shifts and oscillations of the video bitrate requested. Example embodiments of the present disclosure can offer a new rate adaptation algorithm to solve these (and potentially other) problems. The HAS client(s) implementing such an approach would not suffer from the rate shifts/oscillation when they compete for bandwidth at network bottleneck links.
Second, when competing for bandwidth at a network bottleneck link, typical HAS client(s) rely on their underlying TCPs’bandwidth sharing behavior. This may sometimes be undesirable. For example, the resulting bitrate may be unfairly biased against clients with long Round-Trip Times (RTTs). As another example, when a High-Definition (HD) video stream sharesbandwidth with a Standard Definition (SD) stream, the HD stream should purposely have access to more bitrate. Certain embodiments of the present disclosure are able to decouple the stream bitrate selection behavior from the underlying TCP’s. This can enable any number of application scenarios including, but are not limited to, the examples described herein.
Third, in adaptive streaming scenarios, the objective of avoiding video playout stall is generally associated with the responsiveness of downshifting, but not upshifting. Stated in different terms, there is an asymmetry between the two. Therefore, it is desirable to have an asymmetric rate shift behavior, where the downshift ought to be more responsive (or more aggressive in reducing its bandwidth use than upshift is in increasing it) than the upshift. Certain embodiments of the present disclosure are able to achieve this property. It should also be noted that in certain example implementations, the activities outlined herein can be accommodated entirely by a client-side modification to current HAS solutions, and it would not require changes to the network, to the server, etc.
Figure 3 illustratese a simplified block diagram illustrating possible example details associated with one embodiment of the present disclosure;
Turning to Figure 3, Figure 3 is a simplified block diagram illustrating one possible set of details associated with communication system 100. This particular configuration includesHAS client(s) 112 being provisioned with a buffer 302, a processor 304, a memory 306, a rate control function 308, and a target delay controller 310. Buffer 302 can be configured to buffercontent received at a receiver (e.g., HAS client(s) 112). rate control function 308 can be configured to monitor buffer 302 and determine a status of buffer 302. Target delay controller 310 can be configured to monitor the state of the content stream that the receiver (e.g., HAS client(s) 112) is receiving.
In operation, the elements of HAS client(s) 112 can provide a rate adaptationalgorithm that incorporates an additive-increase/multiplicative-decrease (AIMD) mechanism to gradually adjust the average throughput to match the available bandwidth. It should be noted that the AIMD algorithm does not directly adjust the (discrete) video bitrate; instead, it adjusts the average throughput used, which is equal to the segment size divided by the time interval between the beginning of downloading the current segment and the next segment. In addition, it should be noted that, in using such a framework, congestion would be inferred by the reduction of segment downloading throughput (or equivalently, the increase in segment downloading duration). By contrast, in comparable systems, congestion is generally inferred by packet losses. Additionally, the mechanisms of HAS client(s) 112 can fine-tune the interval between consecutive videosegmentdownloads (e.g., using a proportional-integral (PI) controller, which is being represented by target delay controller 310 in Figure 3).
In at least one example, a controller is used for determining the interval between consecutive segment downloads. For example, a proportional-integral controller (PI controller) (associated with control theory) is a special case of a PID controller in which the derivative (D) of the error is not used. Hence, a PI controller would be an optional module for scheduling the segment downloading. Other controllers could also be used without departing from the scope of the present disclosure. Note that any such controller is entirely optional and, accordingly, certain embodiments do not make use of this controller in order to achieve the operations discussed herein. The following equation can be part of such an implementation:
T [ n ^ ] = r [ n ] .tau. y ^ [ n ] + .beta. ( B [ n – 1 ] – B 0 ) ##EQU00001##
This equation is reflected in 408 of FIG. 4. In this instance, beta is positive real number that can control the convergence rate of buffer B[n-1] towards the reference buffer B.sub.0. Additionally, the notations of the equations discussed herein are defined as follows: [0033] x_hat[n]: The target average throughput at downloading cycle n. [0034] x_tilde[n]: The measured downloading throughput of segment n, defined as the segment’sdata size divided by the segment downloading duration (excluding the *off* interval between downloads). [0035] T[n]–The actual inter-download time of cycle n (i.e., the interval between the beginning of downloading segment n and the beginning of downloading segment n+1) [0036] k–The convergence rate in AIMD algorithm. [0037] .tau. (greek letter tau)–the nominal duration of each segment. [0038] w–The AI weight in AIMD. [0039] T_hat[n]–The target inter-download time of cycle n, which may be less or equal to T[n]. [0040] B[n]–The duration of video buffered at the client (in video seconds). [0041] K.sub.P–The proportional gain in PI controller. [0042] K.sub.I–The integral gain in PI controller. [0043] B.sub.0–The referencebuffer duration that the client tries to maintain.
Separately, the framework of HAS client(s) 112 makes use of the additional degree of freedom introduced by the fine-tuning of the download interval to achieve a weighted bandwidth sharing among HAS client(s) that are competing for bandwidth at some bottleneck link (regardless of the fair or unfair sharing of the clients’ underlying TCP behaviors). Intuitively, an HAS client(s)with less bandwidth (for the same video bitrate) would have a longer download interval and vice versa. Additional details associated with these activities are discussed below with reference to several equations, scenarios, and activities that are illustrative of at least some of the embodiments of the present disclosure.
Parts List
100
communication system
102
servers
104
transcoder
106
media storage
108
network
110
intermediate nodes
112
HAS client(s)
200
adaptive streaming environments
302
buffer
304
processor
306
memory
308
rate control function
310
target delay contoller
Terms/Definitions
particular configuration includes
originating video source
property
few seconds
service
coding
stored and live contents
application scenarios
web server
rate
longer download interval
beginning
results
different segments
stream
state
certain embodiments
different terms
target average throughput
broadcast services
live streaming
algorithm
bitrate selection
upshifts
nominal duration
mixed-media offerings
better quality
PI controller
source video
requested content
PID controller
bitrate
activities
reference
segments
adaptive streaming systems
`fragment
downloading cycle n
buffer
downloaded video segments
fetch operations
simplified block diagram
downshifting
adaptive video
such controller
client
next segment
following equation
several equations
video
streaming capabilities
other controllers
rate control function
example
HTTP/TCP/IP
instance
communication system
program
higher-bitrate video segments
significantly larger amounts
multitude
certain example embodiments
function
its bandwidth use
nearly the same, interval
high average video quality
low chance
video quality
reference buffer B
TCP behaviors
network bottleneck link
widely varying network conditions
resulting bitrate
such an approach
additive-increase/multiplicative-decrease
new rate adaptation framework
fine-tuning
at least one example
type
inelastic systems
multi-rate video encoding
bandwidth sharing behavior
download interval
enhancement
content
examples
downshifts
possible adaptive streaming scenario
HD stream
receiver
downloads
new rate adaptation algorithm
classic cable TV
varying bandwidth
K.sub.I–The integral gain
systems
such an implementation
cycle n
freedom
download history
additional details
measured throughput
chunks
buffering
servers
frequent shifts and oscillations
long Round-Trip Times
bottleneck link
particular system
greek letter
HTTP/SPDY/IP
high-quality streaming video
HAS client(s)
example embodiments
framework
AIMD algorithm
rate shifts/oscillation
asymmetry
solutions
measured downloading throughput
intervals
upshift
several significant mechanisms
several potential benefits
positive real number
memory
video seconds
plurality
segment n
objective
bandwidth
common practice
downshift
controller
client-side modification
length
web paradigm
other transports
congestion control
portion
operations
intermediate nodes
high video rates
target delay contoller
network paths
vice versa
various data services
same content
network bottleneck links
basis
interval
clients
different durations
equation
embodiments
plain HTTP GETs
content stream
responsiveness
stream bitrate selection behavior
following foundational information
architecture
effects
notations
HTTP Live Streaming
details
segment
fine-tuned interval
symmetric rate upshift/downshift
reliability
available network bandwidth
streaming information
video playout stall
varying quality
original non-encoded video source
suitable information and/or data
techniques
their underlying TCPs’
same rate
downloading
unfair sharing
client buffer
tens
disclosure
seconds
Standard Definition
access
same duration
part
data size
viable rate adaptation algorithm
rate adaptation
less network bandwidth
RTTs
processor
previous media activity
segment downloading
inappropriate way
program content
different rates
frequent rate shifts
single encoded source
excellent video rate stability
adaptive streaming environments
proportional-integral controller
previous several segments
multiple service providers
segment’s
contrast
media storage
certain example implementations
information
asymmetric rate shift behavior
its underlying TCP’s
transcoder
weighted bandwidth sharing
lower encoding bitrates
unexpected bandwidth fluctuation
probe-adapt principle
scope
figure
low variation
inter-request intervals
typical scenario
High-Definition (HD) video stream shares
performance
data delivery
one embodiment
consecutive segment downloads
available bandwidth
certain content
error
scenarios
elastic IP transport protocol suite
proposed rate adaptation algorithm
programs
additional degree
higher encoding rates
equations
audio, games, applications, channels
average throughput
most adaptive streaming technologies
optional module
possible example details
TCP/IP transport
program time
quite common practice
underlying TCP’s
flow control features
adaptive streaming scenarios
adaptive streaming
term `segment` and `chunk
near real-time
weight
historical data
adaptive streaming video systems
follows
higher encoding rate
same video bitrate
delivery rate
buffer underruns
behavior
case
number
their buffers
multi-rate encoder
multiple
segment downloading duration
convergence rate
AIMD
lower encoding rates
flow’s
underlying transport protocol’s
less bandwidth
clients’
higher or lower encoding rates
multi-rate coding
consequent video stalls
digital media bundles
control theory
fill level
bandwidth challenges
downloading segment n
simultaneous users
environment
multiple content providers
changes
status
T[n]–The actual inter-download time
beta
network
special case
addition
present disclosure
order
changing network conditions
rate oscillation
media player
detailed below
streaming
SCTP
typically hypertext transfer protocol/transmission control protocol/Internet protocol
network server
multiple rates
video bitrate
new time discount addition
server