1

The Joy of Queueing

Part B

Dr. Mark P. Van Oyen

A mixture of queueing basics, analogies, and operations management principles.

Filename: 574queue-lec-B.ppt

2

Waiting Line AnalysisAssists managers in determining:

• Likelihood a customer will have to wait (Prob of nonempty sys)

• Average time a customer will wait (customer average –discrete)

• Average number of customers waiting• (time-average – continuous)• Waiting line space needed (System design … How to address?)

• Percentage of time all servers are idle (time average)

• How many servers we need to employ? (System design … How to address?)

3

Waiting Line Analysis – cont’d

Many other uses and thousands of models and formulas exist – but let us begin with the most fundamental model of queueing theory:

… the M/M/1 queue. The formulas will shape our intuition and give us a knowledge of how congestion behaves.

4

Waiting Line Terminology

Queue - a waiting line + server(s) [we will say “queue only” if we mean to exclude the server]

Number of Servers - the workers or machines that process jobs sequentially (typically one customer at a time) [as a complication that can be addressed via simulation, workers may have different speeds and different skills]

5

Waiting Line Terminology – cont’d

Arrival rate () - (arrivals per unit of time).

M/M/1 - exp(mean 1/ )

Service rate () – (service completions per unit of time).

M/M/1 - exp(mean 1/ )

Queue discipline – tend to emphasize First-come-first-served, which is the default for the M/M/1 queue

6

Beyond the Arrival Rate… distribution!

Analytical models tend to assume that customer arrivals are independent of each other and vary randomly over time.

In simulations, we tend to allow greater fidelity of modeling: customer classes, complex processes generating the arrivals, etc.

For the GI/GI/1 model, the times between arrivals are called interarrival times and their distribution must be specified.

7

Service Times

Service Time = the time it takes to serve a task (uninterrupted). • If the model allows the service period to be interrupted

(e.g., preempted by a high priority customer, or the server takes a break/vacation), then we have two standard cases:

• Preemptive Resume: when service resumes, the service time remaining is computed as total service time minus time already served.

• Preemptive Repeat: when service resumes, the service time remaining is computed as the original total service time (thus time already served is wasted).

M/M/1 model does not have any service interruptions

8

Exponential (Memoryless) Inter-arrivals

When arrivals are quite random and may come from many sources, the interarrival time is often taken to be exponential (time studies are needed to identify the actual distribution).

Exp (mean 1/ ) interarrival times imply that the number of arrivals during t units of time is a random variable with a Poisson (*t) Distribution. When *t is large, this can be approximated as a normal distribution with mean *t and standard deviation = square root(*t).

Equivalently, a Poisson(rate) arrival processes produces Exp (mean 1/ ) interarrival times

9

M/M/1 Model of a Gift Shop

• Single Queue (channel) with a single class of customers

• Infinite waiting room in the queue• The queue discipline is “first-come, first

serve”• Infinite calling population (customer base).

Note: With a finite population, it’s more complicated as every entity must be accounted for at all tiems. This is implicit in the assumption of the Poisson Arrival Process (exp(1/) inter-arrival times).

• Single, always-available server (that’s the 1 in M/M/1)

• exp(mean 1/ m) service times • only one service phase

10

M/M/1 Model of a Gift Shop – cont’d

Must assume something about and µ … What?

11

M/M/1 Model - Stability

Must assume something about and µ

customers are served at a faster rate than they arrive (i.e., < µ ).

If > µ, …

12

13

M/M/1 Model - Stability

Stability if and only if < µ

If > µ, customers will continue to backup and all cannot be served. We call queues with > µ “explosive” or “unstable”.

Note: In simulation, many are fooled by getting large numbers for average queue length. Is that the true average, or simply the fact that that the simulation terminated before the explosion got worse? Instability is not always easy to determine via simulation!

14

Queueing Variables O.M. Terms

For a GI/GI/1 system, = system Throughput (jobs/hour)

(which may be thought of as the average release rate)

µ = average capacity (jobs/hour), which is a maximum potential throughput rate.

15

Definitions–cont’d

=server utilization (long run fraction of time server is busy)

P0 = probability of empty system (arrival does not wait) = 1 -

WIP = L = average number of customers in the system (Work In Process)

Cycle Time (flowtime) =CT = W = average cust. waiting time (queue

+ service)

16

Key Relationships of QueueingFrom the perspective of an arriving customer.Say customer number 3 is the one we want to

watch:

Service time Wait in queue

Arrival time

Cycle time (CT)

17

Key Relationships of Queueing

System perspective on WIP vs time:

Service time of job 3

Arrival time of job 3, the “watched” customer

Cycle time (CT)

1 2 3 4

1

2

3

4

5

5

18

Key Relationships of Queuing-cont’d

Watching the Server:

1. Complete current job

2. Idle if no customers present

3. Take next customer

according to

queue discipline

= arrival event

= departure

busy idlebusy busy

Utilization = fraction of time busy (U or )

19

The probability (P0) that no customers are in the queueing system (either in the queue or being served)

= (1 - =/µ

=/µ = fraction of time server is busy

The average number of customers (L) in the queueing system (being serviced and in line) is

WIP = L = / (1 - )

M/M/1 Queueing Formulae Summary

server utilization = ?

20

The average time a customer spends in the total queueing system - i.e. waiting and being served (W)

The average time a customer spends waiting in the queue to be served (W q).

U = = The probability that the server is busy (i.e. the probability that the customer has to wait). U is the “utilization factor”

CT = W = 1/(µ - )

= /(µ - )

M/M/1 Queueing Formulae

21

Queueing Example: M/M/1

• Jim Bream, the server, pulls stock from his warehouse shelves to fill customer orders.

• Arrival time = time a customer joins checkout line, knowing what they want (not entry through doorway)

22

Queueing Example: M/M/1 –cont’d• The times between arrivals (called

interarrival times) are exponentially distributed with arrival rate 20. Implications:•Customer orders arrive at a mean arrival rate of 20 per hour.

•The number of arrivals in t hours is Poisson distributed with mean 20*t.

• The service time to pull an item for any order is exponentially distributed with a mean of 2 min. (or 2/60 of an hour)

23

Queueing Example

Service Rate from service time distribution

Question

What is Jim’s mean service rate per hour?

Answer

Since Jim can process an order in an average time of 2 minutes (= 2/60 hr.), then the mean service rate, µ, is µ = 1/(mean service time), or 60/2 = 30 orders/hr.

24

Queueing ExampleServer Utilization Factor

QuestionWhat percentage of the time is busy

Jim processing orders (as opposed to being idle)?

Answer

The percentage of time Jim is processing orders is equivalent to the utilization factor, = /. Thus, the percentage of time he is processing orders is:

= / = 20/30

= 2/3 or 66.67% of the time

25

Queueing Example

Average Time in the System

Question

What is the average time a customer must wait from the time they join the checkout line until Jim fulfills the order (i.e. cycle time or waiting time)?

26

Queueing Example

Average Time in the System

Answer

With = 20 per hour and = 30 per hour, the average time an order waits in the system is: CT = W = 1/(µ - )

= 1/(30 - 20)

= 1/10 hour or 6 minutes

27

Queueing Example

Average Time in the Queue (but not in service):

Question

What is the average time a customer must wait from the time they join the checkout line until Jim begins to take their order (i.e. waiting time in queue)?

28

Queueing Example – cont’d

Answer

With = 20 per hour and = 30 per hour, average wait time in queue is:

CTq = Wq = /(µ - ) = (2/3) * 1/(30 - 20)

= 2/30 hour or 4 minutes

Check: W = Wq + 1/ µ = 4 + 2 = 6 minutes.

29

Queueing Example

Average (Mean) Length of Queue

Question

What is the average number of customers waiting in line to be processed?

Answer

Try to reason it out based on the fact that there are 4 minutes of waiting time in queue prior to service. During this time, services of prior customers are occurring.

30

Queueing ExampleAverage (Mean) Length of Queue

Answer

The average number of orders waiting in the system (queue + server) is:

WIP = L = / (1 - ) = (2/3) / (1-2/3)

= (2/3) / (1/3)

= 2 orders

This is a little surprising if you think deeply, but it is a result of memoryless property of exponential service times. The customer currently in service still seems a mean of 2 minutes (despite the “age” of the service already completed) and of course, the person in line averages 2 minutes.

31

Basic M/M/1 Queueing Takeaway

How does congestion (e.g., WIP) grow as the utilization of a server goes to 100%?

What if the system had multiple identical servers? (same answer, curve grows more slowly)

32

WIP (QueueLength) VS Utilization

Avg. Number in Queue (WIP in queue) vs. Utilization (

0

2

4

6

8

10

12

Lq

33

CT (Waiting Time) VS Utilization

Average Wait (CT) vs. Utilization (rho)

0

2

4

6

8

10

12

Rho

W W

40

Little’s Law

LAW: For any “stable” queueing system (even a network of queues)

L = * W (queueing notation), that is,

WIP = TH * CT (production systems notation),

The same relationship holds if we count only the customers and time actually waiting in queue, but not being served:

Lq = * Wq

WIPq = TH * CTq

41

Little’s Law Example: Gas Tanks

Given:

1. An automotive assembly plant produces 300,000 cars per year

2. The length of the assembly line from the first task to the last is 1.5 miles

3. Gas tanks are regularly sent in batches to the plant, stored in inventory, and then used in the assembly operation. On average, the time from gas tank arrival at the plant to the completion of the finished car is 3.5 days.

How much gas tank WIP (raw material or in production) does the plant carry?

42

WIP = TH * CT

(1) TH = 300,000 tanks/year

(3) CT = 3.5 days * (1 year/ 365 days) = .0096 years

WIP = 300,000 tanks/year * .0096 years

= 2,877 tanks

This is a lot, but we are producing 822 cars/day on average.

Little’s Law Example: Gas Tanks

43

Little’s Law Example: Cash FlowGiven:

(1) A firm carries an average annual finished goods inventory (FGI) worth $1 Billion.

(2) Average annual sales is currently $4 Billion.

We focus on cash flow rather than product flow.

What is the average cycle time of a dollar that is put into FGI? Note: this is an aggregate model for a typical product.

44

Little’s Law Example: Cash Flow

Given:

(1) WIP = $1 Billion.

(2) TH = $4 Billion.

What is the average cycle time of a dollar that is put into FGI?

WIP = TH * CT

1 B. = 4 B. * CT

so the CT = WIP / TH = 1/4 year

CT = 3 mo's on average that our dollar

spends sitting in inventory.

48

Capacity Utilization vs. Customer Waiting in Services

Cost or inconvenience of waiting is usually measured by • waiting time• Blocked customers (lack of waiting space – get “busy signal”)

• Customer reneging or abandonment (call center)

• Fraction cust. served within X minutesResult:

• Higher utilization of service facilities • Is gained at price of customer waiting

49

Queuing Models and Capacity Planning

Have capacity – service level tradeoff

Goal of queueing system global optimization is: minimize total costs of • service capacity • +• waiting

What does a curve of total cost versus capacity to serve look like?

50

Analysis of Costs in Congested Systems

$

Service Level

Total Cost

Cost of providing service

Cost of customer waiting

MIN COST LEVEL OF SERVICE

Low utilization, High utilization,

51

What About Variability?

In an M/M/1 queueing model such as we have studied, the amount of variability has been fixed. I.e., regardless of the and we pick, the variability is constant (coefficient of variation (c.v.) = (std. dev. / mean ) = 1.0). In general 0 < c.v. <

The amount of variability is captured in

1. The interarrival distribution: c.v. = CG

2. The service time distribution: c.v. = CX

How does increasing variability impact the system performance?

52

What About Variability? M/G/1 Model

Most common application: M/G/1

1. The interarrival distribution is EXPONENTIAL, c.v. = CG =1 and the arrival process is called a Poisson Process.

2. The service time distribution has mean 1/ but the distribution has more or less variability: 0 < CX <

53

What About Variability? M/G/1 Model-cont’d

Result: Wait in queue only = ½ (1 + CX2) Wq ,

Where in the M/M/1 model we defined Wq = /(µ - )

What does a curve of waiting time in queue versus service time variability look like?

54

M/G/1 Model has congestion roughly proportional to the square of std. dev. of service/process time

Cx Lq Wq Wq of M/M/10 0.9 0.5 4.1 9.0 18

0.33 4.5 10.0 180.67 5.8 13.0 181.00 8.1 18.0 181.33 for this model, 11.2 25.0 181.67 lambda = 0.45 15.3 34.0 18

2.00 20.2 45.0 18

2.33 26.1 58.0 182.67 32.8 73.0 183.00 40.5 90.0 183.33 49.0 109.0 18

Get Wq

Average Queue length vs. Coef. Var. Service Time

0.0

10.0

20.0

30.0

40.0

50.0

60.0

00.

671.

332.

002.

673.

33

Coef. Var. Service time

Lq

Average Waiting Time vs. Coef Var Service Time

0.0

20.0

40.0

60.0

80.0

100.0

120.0

00.

671.

332.

002.

673.

33

Coef Var Svc. Time

Wai

t in

Qu

eue

Utilization is constant= 0.90.

Taken from

queue-approx.xls

55

Variability Principles

1. If, in addition, we increase the variability in the interarrival distribution: CG > 1, then we also get an increase in Queue length and waiting time (WIP and CT) that is roughly proportional to the square of the standard deviation of the interarrival time distribution. (This is from the G/G/1 model, which can only be approximated).

56

Role of Simulation

• As we depart from exponential distributions, analysis becomes more difficult and simulation is needed in many cases

• Simulation also provides the ability to model systems with • greater fidelity to the current reality

• Or greater complexity

IN CONTRAST, mathematical analysis is well-suited to fundamental design/evaluation and the extraction of general principles.

• In Operations, simulation is often used as a research tool to perform sensitivity analysis (e.g., showing if the insights remain valid when variability departs from the CV of 1 constrained by the exponential distribution)

57

Variability Principles – cont’d

• End result: congestion is roughly proportional to the variances of the system – interarrival and process (service) time. (i.e. proportional to standard deviations SQUARED).

• Variability reduction is REALLY IMPORTANT. Cutting process time by 50% to /2 can result in a CT reduction to as low as CT/4 – even without any reduction in total, mean processing time.