A QoS Control Method Cooperating with a Dynamic Load Balancing

1

A QoS Control Method Cooperating with a Dynamic Load Balancing Mechanism

Akiko Okamura, Koji Nakamichi, Hitoshi Yamada and Akira Chugo

Fujitsu Laboratories Ltd.4-1-1, Kamikodanaka, Nakahara, Kawasaki, 211-8588, Japan

Telephone: +81-44-754-2635 Fax: +81-44-754-2741E-mail address: {akikoo, nakamichi, hitoshi, chugo}@flab.fujitsu.co.jp

AbstractA dynamic traffic engineering architecture is described that provides QoS-guaranteed service

(GS) in addition to existing best effort (BE) service. It is designed to provide bandwidth-guaranteed paths for GS traffic along optimum routes and to provide one or more paths for BE service traffic along routes that will best avoid congestion. Simulation showed that the proposed routing algorithm achieves both load balancing and efficient resource usage and that dynamic load balancing improves the throughput of BE traffic. Use of this architecture will help to provide QoS-guaranteed service, effectively utilize network resources, and avoid degradation in BE traffic throughput.

Keywords:IP Network Management, Traffic Engineering, MPLS, Load Balancing, QoS, Network Optimization

2

Fujitsu Laboratories Ltd. 2APNOMS2003

Introduction• Current IP network problems

– End user's viewpoint• Degraded of performance (e.g., lowered throughput and increased

delay) due to congestion– Network operator's viewpoint

• Profits do not improve though traffic increases every year• Frequent bandwidth increases needed to support traffic increases• Services supporting usage-based billing are limited

• To overcome these problems– Provide QoS guaranteed service– Develop fee/charge system based on network QoS – Use network resources efficiently--> IP traffic control mechanism is required

1. IntroductionAs the number of users accessing the Internet through broadband connections continues to

accelerate and the needs of the users become more diversified, quality of service (QoS) guarantees are becoming more important for both Internet service providers (ISPs) and end users. Since only best effort (BE) service is now provided, network performance is sometimes degraded by congestion. Moreover, even though the amount of Internet traffic is increasing, the providers are still not making much profit. This is because frequent increases in bandwidth are needed, and optional services that support usage-based billing are limited. To overcome these problems so that we can build a new network based on Internet technologies as an infrastructure for the coming broadband, ubiquitous era, we need to develop a QoS control system and a fee/charge system based on network QoS [1]. Doing this will enable ISPs to provide usage-based billing services and enable end users to enjoy reliable networks. Furthermore, network resources should be used efficiently to avoid installing too many network facilities. All of this requires that networks have an IP traffic control mechanism.

In short, our challenge is to provide QoS guarantees without over-investing in network facilities. Additionally, new network services should come with a range of QoS-guaranteed service, such as bandwidth guarantees and delay guarantees.

3


Our Approach to IP Traffic Control

QoS: quality of serviceSLA: service level agreementSPF: shortest path first

Leve

l of

netw

ork

effic

ienc

y

TrafficEngineering (TE)

Minimum hop routing

+Explicit routing

+Multi route&

path control

+Dynamic flow splitting

BestEffort

＋

Packet-priority

mechanism

＋Admissionpolicingshaping □

＋Performance monitoringreporting

＋

Account-refund

Connectionless approach(Diffserv)

Connection-orientedapproach(IntServ)

Explicit path setup

Static load balancing

Dynamic load balancing

Low High

Low

High

Level of QoS/SLA

Future

Hop-by-hopforwarding

Proposed method

2. Our Approach to IP Traffic ControlThis diagram illustrates our approach.The final goal, shown on the horizontal axis, is to offer various service level agreements (SLAs).

Simply introducing a QoS mechanism is not enough. End users may want to know the actually provided QoS so they can compare it with the target in the SLA. ISPs may have to refund service fees if the target is not achieved.

The vertical axis shows how much freedom and flexibility there is in selecting the route. In legacy IP networks, a single route between the ingress and egress routers is selected based on a single criterion, such as shortest path first. This can result in traffic congestion and does not support flexible routing. A network operator achieves its netowork operation policy by using constraint-based routing. Reliable network operation can be achieved by using two or more routes between the ingress and egress routers. However, the static use of two or more routes works poorly in dynamic network situations such as congestion due to burst traffic or faults. Dynamic control can improve performance, for example, by moving traffic from a congested route to one or more non-congested routes. A network should be used as a two-dimensional space, not as a linear line, to improve its tolerance of such dynamic situations.

As shown in the figure, several technologies, such as differentiated services (diffserv) [2] and integrated services (intserv) [3], provide QoS and traffic engineering techniques that enable the effective use of networks. However, the use of these techniques does not satisfy the goals of offeringQoS and effectively using networks simultaneously. We thus developed an IP traffic control method based on traffic engineering that can provide guaranteed QoS while using network resources efficiently.

4


Traffic Engineering (TE)• Description

– Improves traffic performance– Facilitates reliable network operations– A main application of multiprotocol label switching (MPLS)

Constraint-based routing (explicit routing)

• Example Applications– Static/dynamic load balancing

• Achieves highly reliabile network by avoiding congestion/failure• Enables efficient use of bandwidth resources--> Functions of dynamic load balancing have been proposed

– Fast RerouteAchieves highly reliabile network through high-speed failure

recovery

3. Traffic Engineering (TE)Traffic engineering is used to improve traffic performance and to effectively use network

resources by avoiding network congestion [4]. It is one of the main applications of multi-protocol label switching (MPLS) [5]. In particular, it enables network congestion to be avoided by using constraint-based routing. A label switched path can be flexibly established not only along the shortest route but also along the route with the largest residual bandwidth or a route that satisfies the network operator’s policy(e.g., prohibit to transit a particular node).3.1 Example Applications

Traffic engineering is commonly used for static and dynamic load balancing and fast rerouting. Static and dynamic load balancing use multiple routes to a destination to avoid congestion/failure. For static load balancing, traffic is split statically based on a predefined ratio. For dynamic load balancing, it is split based on the actual traffic load. Fast rerouting establishes detour paths in case of a failure. If the network detects a link or node failure, it quickly moves traffic from the original route to a new route on the detour path. Using fast rerouting, a network can achieve high reliability through high-speed failure recovery.3.2 Dynamic Load Balancing

Dynamic load balancing has been proposed as an application of dynamic TE [6]. It optimizes network resource usage and maximizes network throughput. The main functions of network load balancing are - traffic statistics monitoring, - detour route establishment, - explicit path setting, and - load balancing. The load balancing algorithms are described elsewhere [7].

5


Basic Architecture of Proposed Method- Cooperation between dynamic load balancing and providing QoS guarantee -

TE Controller

QoS path control for guaranteed class traffic

QoS pathBandwidth reservedHigh priority at schedulingOptimum route considering both network and application server resources MPLS: multi-protocol label switching

MPLS Network

Best Effort pathOriginal path = minimum hop routeDetour route used when there is congestion

Application Server

Statistics monitoring(Network, Application servers)

User

QoSrequest

Load balancing control for best effort traffic

Admission control

4. Proposed method4.1 Basic Architecture

Our proposed QoS control architecture works in cooperation with a dynamic load balancing mechanism. Here, we consider bandwidth to be the QoS metric. The network provides services via bandwidth-guaranteed paths (QoS paths) using MPLS.

The architecture achieve efficient use of network resources, which helps avoid over investment in infrastructure, by providing QoS paths along the optimum routes for the guaranteed service (GS) traffic and by providing one or more paths along routes that will best avoid congestion for the best effort (BE) service traffic.

This diagram shows a centralized architecture using a TE controller, one approach to implementing our proposed method. We assume that BE and GS traffic share the network resource (i.e., link bandwidth). The TE controller maintains the network topology, monitors the network and server statistics, accepts user requests, and sets label-switched paths to routers for GS and BE traffic. Each output interface of each router in the network has a priority queue as a scheduling mechanism; it sends GS traffic packets at a higher priority.

The path for GS traffic is calculated as an optimum route in order to use both the network and application server resources efficiently. The calculation is done using a topology graph constructed from links satisfying the requested bandwidth, and that bandwidth is reserved on each link along the path. When a single path accommodates aggregated flows, each user request must pass through admission control. If the bandwidth of the path is insufficient to accommodate more flows, an additional path is established. Since bandwidth is not reserved on each link along a path for BE traffic, the shortest route is selected for BE traffic using the entire network topology graph. However, if the route becomes congested, an alternative path is set up and the load is dynamically balanced.

As an example application of our proposed method, we will describe a contents delivery network providing a streaming service with a QoS guarantee. Multiple streaming servers are connected to edge routers, and the streaming contents are distributed among these servers. End users can request the contents with the QoS guaranteed or with best effort; with the QoS guaranteed, users pay an extra charge for a guaranteed bandwidth. The TE controller accepts the request, selects the optimum streaming server and route, sets up the QoS path, and directs the user’s request to the selected server.

6


QoS Routing Algorithm• Minimize total cost of link and server

total cost = server cost + link cost

server cost: 1/(residual available output rate)link cost: 1/(residual available bandwidth)

link costserver cost

User

TOTAL COST is MINIMUM!

Link D-E cost is high…

A

B

CD

E

server cost is high…

4.2 QoS Routing AlgorithmThe basic idea of our algorithm for selecting the optimum server and route for guaranteed QoS is

to assign a cost to each link and to each application server based on its load (the higher the load, the higher the cost) and to search for the route and the server that minimize the sum of the costs of the links along the route and the cost of the server.

Link cost is based on the non-reserved bandwidth of the link, so we can definelink cost = 1 / (R_MAX – R ), (1)

where R_MAX denotes the link’s maximum reservable bandwidth and R denotes the bandwidth actually reserved.Server cost represents the load on the server based on its bottleneck (CPU, IO, etc.). Several

values can be used to represent the cost: CPU load, memory consumption, disk access load, number of user connections, traffic output rate, etc. A streaming server, for example, generally has an IO or NIC bottleneck rather then a CPU load bottleneck, so we can define

server cost = 1/(S_MAX – S), (2)where S_MAX denotes the maximum output rate (NIC rate) and S denotes the current output rate.

After determining the cost of each server and each link, we search for the route and the server that minimize the total cost of the links along the route and of the server. We calculate the minimum cost route to each candidate server, from the server's edge node to the user's edge node, using the Dijkstra algorithm. We then select the set of server and route with the minimum total cost. We thus balance both the server load and the network load at a same time, using resources efficiently.

7


Evaluation of QoS Routing:Metrics and Model• Metrics

– Number of QoS requests accepted – Average number of hops in QoS pathsCompared with

• LSL method(lowest server load)

• DNS method(domain name server)

• Simulation model– ISP network, 19 nodes– Application servers

• Four• Capacity of 500 Mbps

– QoS requests• 1-10 Mbps bandwidth guarantee (random)• User’s edge selected at random (user’s edge ≠ server’s edge)

OC3 (155 Mbps)

T3 (45 Mbps)

Server locationcandidate

Select server with lowest load

Select minimum cost route to server

Select nearest server

1st step

2nd step

5. Evaluation5.1 QoS routing

We evaluated the effectiveness of the proposed server and route selection algorithm for QoS-guaranteed service by simulation. The metrics were the number of requests accepted and the average number of hops in the QoS paths. For reference, we also simulated the performance of two methods that can be achieved by combining existing techniques.

- LSL (lowest server load) method: 1) select server with lowest load; 2) select minimumcost route to server.

- DNS (domain name server) method: 1) select nearest server (based on location of user’s edge); 2) select minimum cost route to server.

The criterion for route calculation with both methods was the same as with our proposed method—available bandwidth. 5.1.1 Model

We used an ISP network model with 19 nodes [8], as shown in this slide. The network consisted of 155- and 45-Mbps links. There were four servers; the edge routers connected to them are selected at random from the routers shown as slash-pattern circles. End users connected with the other edge routers and requested QoS paths with a specified bandwidth requirement. For each request, a user's edge was selected at random and the requested bandwidth was selected randomly (between 1 and 10 Mbps). The capacity of the server (i.e., the maximum transmission rate) was 500 Mbps. For simplicity, QoS paths were not released.

8


Evaluation of QoS Routing: Results• Effect of QoS routing considering both server and network

loads– Accommodates many more requests– Provides QoS path with the smallest number of hops

0

100

200

300

400

Number of QoS requests

Nu

mbe

r of

req

ues

ts a

ccep

ted

0 200 400 600 8000 200 400 600 800 Number of QoS requests

2.8

3.0

3.2

2.0

2.6

2.4

2.2

1.8Ave

rage

nu

mbe

r of

hop

s

DNS method

Our Proposal

LSL method

5.1.2 ResultsThese graphs show the results.The one on the left shows the number of accepted requests as a function of the number of

requests. More requests were accepted with our method. With the DNS method, a user's request is rejected when there is no available route to the nearest server, even if a route to another server is available. With the LSL method, the lowest load server is selected first even if the route to the server is long. A lot of link bandwidth can thus be consumed unnecessarily, limiting the number of requests accepted. With our method, a medium load server to which there is a short route can be selected, even if there is a lower load server with a longer route to the server. Moreover, when there is a short route to a server with a high load, that combination of server and route is unlikely to be selected with our method. In short, our method flexibly selects a server and route based on both the server and network conditions, resulting in a lower blocking probability.

The graph on the right shows the average number of hops as a function of the number of requests. The number of hops was the lowest with our method, meaning that network resources were used more efficiently. With the LSL method, long routes tend to be selected, as we mentioned above. It is interesting that even the DNS method, which selects the nearest server, suffered an increase in the number of hops as the number of requests increased. This is because a longer detour route is likely to be selected if the link utilization of the shortest route to the nearest server becomes high. With the DNS method, the longer route tends to be selected even if there is a shorter route to another server, so the number of hops is higher than with our method.

The average link utilization after 800 requests were received was 43.6% with our method, 50.3% with the LSL method, and 49.2% with the DNS method. Our method can thus achieve a lower load even if it accepts more requests. In this way, our method optimizes the network to achieve efficient resource utilization.

9


Evaluation of Dynamic Load Balancing:Metric, Model, and Conditions

• Evaluate effect of dynamic load balancing under GS traffic conditions

• MetricThroughput of BE traffic

• Model5-node-ring model with 100 Mbps links

• Conditions– 50 Mbps BE traffic– Bandwidth reserved for GS

traffic is increases to 80 Mbps.– Actual amount of GS traffic fluctuates– Congestion detection conditions

A) Actual (GS+BE) traffic > 80 Mbps B) (Reserved GS + Actual BE) traffic > 80 Mbps

100 Mbps

BE: 50 Mbps

GS

Reserved

Actual GS traffic

5.2 Dynamic load balancingWe evaluated how well dynamic load balancing (D-LB) avoids performance degradation of BE

traffic due to GS traffic with a time-varying rate. The performance of BE traffic is easily degraded due to actual GS traffic because GS packets have higher priority, as mentioned above. Dynamic load balancing can prevent this degradation. The metric for this evaluation was the throughput of BE traffic.5.2.1 Model

To show the effect simply, traffic flowed between two points connected by two paths. The link capacity was 100 Mbps, the maximum reservable bandwidth for QoS traffic was 80 Mbps, and the input BE traffic rate was fixed at 50 Mbps. GS flows were added at intervals of 5 seconds with the bandwidth reserved for GS increased in 6-Mbps steps. The actual amount of GS traffic varied based on the sine function.5.2.2 Congestion detection conditions

When congestion was detected under one of two conditions, load balancing was triggered. Under condition (A), the total amount of actual GS and BE traffic in a link exceed the threshold parameter, the “condition detection threshold (CDT),” which was set to 80 Mbps. Under condition (B), the sum of the bandwidth reserved for GS traffic and the actual amount of BE traffic exceed CDT. Dynamic load balancing was initiated when the specified condition continued for more than 30 seconds.

If condition (A) was specified, dynamic load balancing was done based on the actual state of the GS traffic. This means that if there was enough available bandwidth along the shortest route (because the total amount of actual GS traffic was less than the bandwidth reserved for GS), BE traffic could be sent using the shortest route.

If condition (B) was specified, BE flows were moved to the detour path by dynamic load balancing when the bandwidth reserved for GS traffic was increased. In this case, degradation in BE traffic performance was avoided even when the actual GS traffic rate suddenly reached 100% of the reserved bandwidth.

10


Evaluation of Dynamic Load Balancing: Results

Time [s]

Thro

ugh

put

[Mbp

s]

Actual GS trafficActual GS traffic

BE traffic with D-LBD-LB1 (condition A, solid line)D-LB2 (condition B, × signs)

BE traffic with D-LBD-LB1 (condition A, solid line)D-LB2 (condition B, × signs)

BE traffic without D-LBBE traffic without D-LB

Bandwidth reservedfor GS traffic

Throughput decreases because load cannot be moved to other available links.

Throughput decreases because load cannot be moved to other available links.

•Input BE Traffic= 50 Mbps (fixed)

Throughput remains almost maximum.

Throughput remains almost maximum.

•D-LB:Dynamic load balancing

5.2.3 ResultsThis graph shows throughput as a function of time. The throughput of BE traffic without D-LB

(dynamic load balancing), with D-LB under condition (A) (D-LB1), and with D-LB under condition (B) (D-LB2) are shown in addition to the bandwidth reserved for GS traffic and the actual throughput of GS traffic.Without D-LB, BE traffic throughput degraded when the GS traffic rate exceeded 50 Mbps. In contrast, with D-LB, BE traffic throughput remained almost always at 50 Mbps. With D-LB1,

the throughput decreased for a while due to a lag in detecting the congestion caused by the increase in the actual GS traffic.With D-LB2, congestion was detected immediately, so there was no degradation in throughput of

BE traffic.Dynamic load balancing prevents degradation in the performance of BE traffic because it uses

other available links efficiently. If the actual rate of GS traffic fluctuates intensely, it is safe to use condition (B) to avoid degradation in BE traffic performance.

11


Implementation

Path-setting status display

- Snapshot of Operation Screen -

Click

Control statusdisplay panel(Path setting, load balancing, etc.)

Detailed path information

6. ImplementationWe implemented a prototype traffic engineering(TE) controller capable of dynamic load

balancing and QoS path setting using software under SunOS. It has the following functions.- Admission control: accept end user service requests with bandwidth guarantee if they can

be satisfied using existing resources- QoS control: calculates optimum route for GS flow- Dynamic load balancing control: calculates detour route for BE traffic when congestion is

detected; if there are multiple available paths for BE traffic, periodically calculates, based on network situation, ratio of flows to be moved to each path

- Statistics monitoring: periodically monitors statistics for each router and each application server

- Router setting: controls each router via a command line interface (CLI) to set up label switched paths, accommodate GS flows into each path, and split BE flows into multiple paths

As an actual application for QoS (bandwidth) guaranteed service, we selected video-on-demand service. Users request streaming content with a bandwidth guarantee from a contents menu screen on a Web server. The TE controller calculates the optimum route on acceptance of the user’s request, considering the hop count, available bandwidth, and streaming server load. After determining the route, the controller sends commands to the ingress router via the CLI to set up the QoS path for the requested flow and then responds to the user via the Web server screen. Concurrently, the controller collects network information from the routers by using SNMP and maintains the information as a database of the topology, paths, and statistics. Based on this information, the controller detects congestion, executes route search, and balances the BE traffic load by sending commands to the edge routers.

We built a test-bed system using this prototype TE controller. The behavior of the system was monitored using an operation GUI we developed. The GUI shows the control status of the controller such as “congestion is detected” or “QoS path searching.” It also shows the network topology, the statistics collected from the routers, and the path information for GS and BE traffic.

12


Conclusion & Future WorkConclusionProposed method effectively utilizes resource while providing QoS• QoS routing based on network and server loads

– Number of QoS requests accepted is improved – Server and network load balancing are achieved

• Use of dynamic load balancing effectively provides QoS-guaranteed service– Degradation in BE traffic throughput when GS traffic is fluctuating is avoided

Future work• Evaluation of performance in large-scale network• Development of more advanced QoS control method based on TE

7. Conclusion In this paper, we proposed an architecture and a method for dynamic traffic engineering that both

sets a QoS path along an optimum route for guaranteed service traffic and performs dynamic load balancing for best effort traffic. The result is efficient use of network resources and provision ofQoS-guaranteed services. Simulation showed that our QoS routing algorithm achieves both load balancing and efficient resource usage and that dynamic load balancing improves the throughput of BE traffic. Introducing a dynamic load balancing function that works with a QoS control mechanism when QoS-guaranteed services are provided offers several advantages to both providers and end users.

We plan to study and evaluate the scalability of our architecture and to develop an advanced QoScontrol mechanism based on TE techniques. References [1] Paul Ferguson and Geoff Huston, “Quality of Service: Delivering QoS on the Internet and in Corporate Networks,” John Wiley and Sons, 1998.[2] S. Blake, et al., “An Architecture for Differentiated Services,” RFC2475, IETF, December 1998.[3] R. Braden, D. Clark and S. Shenker, “Integrated Services in the Internet Architecture: an Overview,” RFC 1633, IETF, June 1994.[4] D. Awduche, et al., "Overview and Principles of Internet Traffic Engineering," RFC3272, IETF, May 2002.[5] D. Awduche, et al., “Requirements for Traffic Engineering Over MPLS,” RFC2702, IETF, September 1999.[6] H. Yamada, K. Nakamichi, M. Fukazawa and M. Katoh, "Dynamic Traffic Engineering for Network Optimization - Architecture and Evaluation," Proceedings of APNOMS2002, pp. 270-281,Jeju Island, Korea, September 2002.[7] K. Nakamichi, H. Yamada and A. Chugo, "Dynamic Traffic Engineering under different traffic patterns," 16th International Workshop on Communications Quality & Reliability (CQR2002), Okinawa, Japan, May 2002.[8] Q. Ma and P. Steenkiste, "On path selection for traffic with bandwidth guarantees," Proceedings of International Conference on Network Protocols (IEEE ICNP97), pp. 191-202, Atlanta, USA, October 1997.

A QoS Control Method Cooperating with a Dynamic Load Balancing

Documents

Transcript of A QoS Control Method Cooperating with a Dynamic Load Balancing