This article was downloaded by: [California Poly Pomona University]On: 25 November 2014, At: 01:14Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: MortimerHouse, 37-41 Mortimer Street, London W1T 3JH, UK

Stochastic ModelsPublication details, including instructions for authors and subscription information:http://www.tandfonline.com/loi/lstm20

Workload Process, Waiting Times, and Sojourn Timesin a Discrete Time MMAP[K]/SM[K]/1/FCFS QueueQi-Ming HE a b ca Department of Industrial Engineering , Dalhousie University , Halifax, Nova Scotia,Canadab School of Economics and Management , Tsinghua University , Beijing, Chinac Department of Industrial Engineering , Dalhousie University , 5269 Morris St., Halifax,Nova Scotia, B3J 2X4, CanadaPublished online: 16 Feb 2007.

To cite this article: Qi-Ming HE (2004) Workload Process, Waiting Times, and Sojourn Times in a Discrete Time MMAP[K]/SM[K]/1/FCFS Queue, Stochastic Models, 20:4, 415-437, DOI: 10.1081/STM-200033099

To link to this article: http://dx.doi.org/10.1081/STM-200033099

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) containedin the publications on our platform. However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose ofthe Content. Any opinions and views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be reliedupon and should be independently verified with primary sources of information. Taylor and Francis shallnot be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and otherliabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to orarising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematicreproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Workload Process, Waiting Times, and Sojourn Times in aDiscrete Time MMAP[K ]/SM[K]/1/FCFS Queue

Qi-Ming HE*

Department of Industrial Engineering, Dalhousie University, Halifax,

Nova Scotia, CanadaSchool of Economics and Management, Tsinghua University, Beijing, China

ABSTRACT

In this paper, we study the total workload process and waiting times in a queueingsystem with multiple types of customers and a first-come-first-served service

discipline. An M=G=1 type Markov chain, which is closely related to the totalworkload in the queueing system, is constructed. A method is developed forcomputing the steady state distribution of that Markov chain. Using that steady

state distribution, the distributions of total workload, batch waiting times, andwaiting times of individual types of customers are obtained. Compared to theGI=M=1 and QBD approaches for waiting times and sojourn times in discrete

time queues, the dimension of the matrix blocks involved in the M=G=1 approachcan be significantly smaller.

Key Words: Queueing systems; M=G=1 type Markov chain; Waiting times;

Sojourn times; Workload; Mean-drift method; Matrix analytic methods;Ergodicity; Semi-Markov chain.

Mathematics Subject Classification: Primary 60K25; Secondary 60J10.

*Correspondence: Qi-Ming HE, Department of Industrial Engineering, Dalhousie University,

5269 Morris St., Halifax, Nova Scotia B3J 2X4, Canada; Fax: (902) 420-7858; E-mail: Qi-Ming.He@dal.ca.

STOCHASTIC MODELS

Vol. 20, No. 4, pp. 415–437, 2004

415

DOI: 10.1081/STM-200033099 1532-6349 (Print); 1532-4214 (Online)

Copyright # 2004 by Marcel Dekker, Inc. www.dekker.com

Dow

nloa

ded

by [

Cal

ifor

nia

Poly

Pom

ona

Uni

vers

ity]

at 0

1:14

25

Nov

embe

r 20

14

ORDER REPRINTS

1. INTRODUCTION

Discrete time queueing systems are of special interest to the performance analy-sis of telecommunications systems and manufacturing systems. For instance, discretetime queues are suitable for analyzing ATM in telecommunications (see Alfa[1];Cortizo et al.[4]; and references therein). Therefore, the study of discrete time queueshas attracted great attention from researchers and practitioners.

In this paper, we introduce and study a discrete time queueing model withmultiple types of customers. Such a queueing model arises naturally from moderntelecommunication networks that are required to handle different types of data fromtelephone service, video conferences, high volume file transfer service, high definitionTV distribution service, etc. B-ISDN is such an example. For such systems, differentservices may have different requirements in terms of delay times and loss rates. Toquantify quality of service (QoS) for individual services, there is a need to differenti-ate them and to carry out performance analysis at the individual service level.

For queueing models with a last-come-first-served (LCFS) service discipline, theanalysis of waiting time, queue strings, and busy periods can be carried out throughthe use of Markov chains with a tree structure (e.g., HE and Alfa[13]). However, theanalysis of the queue strings in queueing models with a first-come-first-served(FCFS) service discipline is difficult, since we must keep track of the type of eachcustomer in queue. Nonetheless, significant progress has been made for continuoustime queueing models with multiple types of customers (e.g., HE[9,10]; HE andAlfa[13]; Takine[23,24]; Takine and Hasegwa[25]). The study of discrete time queueswith multiple types of customers is limited, except for Van Houdt andBlondia[26–28]. In this paper, we analyze a discrete time queueing model with a singleserver, multiple types of customers, and a FCFS service discipline. The focal point ison the waiting times of individual types of customers. As is the situation in thecontinuous case, by constructing an M=G=1 type Markov chain associated withthe total workload in the system, we are able to find the steady state distributionof the total workload in the system at an arbitrary time. This enables us to finddistributions of the waiting times and sojourn times for batches and for individualtypes of customers.

The study of waiting times in queues is extensive (e.g., Alfa[1]; Cohen[3];Grassmann and Jain[8]; Neuts[18]; Sengupta[21]; Takine[24]; Takine and Hasegawa[25]

and references therein). For discrete time queues, a number of methods have beenused to study waiting times. Grassmann and Jain[8] used the Wiener–Hopf factoriza-tion method to develop an algorithm for computing the distribution of waiting times.Recently, the matrix analytic methods have been used to study discrete time queues(Alfa[1]; Van Houdt and Blondia[26–28]; Yang and Chaudhry[30]; etc.). Such methodstake advantage of the structures of the Markov chains under consideration anddevelop efficient algorithms for computing system performance measures. The struc-tures of interest include the M=G=1 type and GI=M=1 type (Neuts[17,19]). In VanHoudt and Blondia[26–28], GI=M=1 type Markov chains and QBD processes wereconstructed for studying the waiting times and sojourn times of discrete time queueswith a Markov arrival process with marked transitions and PH-distributed servicetimes. In HE[11], a GI=M=1 Markov chain is constructed for waiting times andsojourn times of a discrete time queue with a semi-Markovian arrival process and

416 HE

Dow

nloa

ded

by [

Cal

ifor

nia

Poly

Pom

ona

Uni

vers

ity]

at 0

1:14

25

Nov

embe

r 20

14

ORDER REPRINTS

PH-distributed service times. In HE[12], a GI=M=1 Markov process is constructed forwaiting times and sojourn times of a continuous time queue with a semi-Markovianarrival process and PH-distributed service times. In this paper, the basic tools arematrix analytic methods. We formulate an M=G=1 type Markov chain to obtainthe steady state distributions of waiting times and sojourn times.

TheM=G=1 type Markov chains have been used in the study of various queueingmodels. For instance, the classical M=G=1 queue and its variants, the MAP=G=1queue and its variants can be analyzed efficiently by using an M=G=1 type Markovchain. The basic theory concerning M=G=1 type Markov chains can be foundin Gail et al.[6], Latouche and Ramaswami[15], Neuts[17,19], and Ramaswami[20]. InTakine[22], a continuous time version of the M=G=1 type Markov process was intro-duced and investigated. As an application, the process was used in the study of thetotal workload and waiting times of a continuous time MAP=G=1 queue. This paperconsiders an extension of the idea used in Ref.[22] from the continuous time case tothe discrete time case. The major differences are (1) we consider a discrete timequeueing model; (2) we consider a queueing system with multiple types of customers;and (3) the M=G=1 type Markov chain in this paper has no boundary at the levelzero.

The approach developed in this paper – anM=G=1 approach – is for queues withgeneral service times, where the methods (GI=M=1 or QBD) developed in HE[11] andVan Houdt and Blondia[26–28]) does not apply. Even for the MMAP[K ]=PH[K ]=1queue, for which all three methods apply, the M=G=1 approach has its advantages.As was pointed by Van Houdt and Blondia[28], the dimensions of the matricesinvolved with the GI=M=1 and QBD approaches can be quite large, which increasescomputation time and space requirement significantly. The size of transition blocksin the M=G=1 approach can be significantly smaller.

The rest of the paper is organized as follows. In Sec. 2, the discrete timeMMAP[K ]=SM[K ]=1 queue is introduced. In Sec. 3, an M=G=1 type Markov chainassociated with the total workload is defined and its ergodicity condition is found.The steady state distributions of the Markov chain, the total workload, waitingtimes, sojourn times of batches of customers are given in Sec. 4. In Sec. 5, two specialcases–the MMAP[K ]=GI[K ]=1 queue and the MMAP[K ]=PH[K ]=1 queue–are consid-ered and more detailed results are obtained. Finally, in Sec. 6, numerical examplesthat provide insight into the performance of the queueing models of interest arepresented.

2. THE DISCRETE TIME MMAP[K ]/SM[K ]/1/FCFS QUEUE

The queueing system of interest has K types of customers, where K is a positiveinteger. All customers, regardless of their type, join a single queue and are served bya single server on a first-come-first-served (FCFS) basis. In the reminder of this sec-tion, we describe the customer arrival process, the queueing process, and the serviceprocess in detail.

The Customer Arrival Process. The customer arrival process is a discrete timeMarkov arrival process with marked transitions (MMAP[K ]) (Asmussen and

A Discrete Time MMAP[K]/SM[K]/1/FCFS Queue 417

Dow

nloa

ded

by [

Cal

ifor

nia

Poly

Pom

ona

Uni

vers

ity]

at 0

1:14

25

Nov

embe

r 20

14

ORDER REPRINTS

Koole[2]; HE[9,10]; HE and Neuts[14]). Customers are distinguished into K types andarrive in batches. To characterize the batches of customers, we introduce a set ofstrings of integers denoted by

@ ¼ fJ : J ¼ j1j2 � � � jn; 1 � ji � K; 1 � i � n; n � 1g: ð2:1Þ

For the queueing system, the string J ¼ j1j2 � � � jn 2 @ represents a batch with n

customers. These n customers are of the types j1; j2; . . . ; and jn, respectively. Withinthe batch, the n customers are ordered as j1; j2; . . . ; and jn. We call J a stringrepresentation of that batch.

The system status is observed at integer epochs t ¼ 0; 1; 2; . . . : Let IaðtÞ be thephase of the underlying Markov chain of the arrival process in the time period½t; tþ 1Þ, which will be called period t, and XðtÞ be the string representation of thebatch associated with the transition in period t if there is an arrival; otherwise,XðtÞ ¼ 0. Then we define

P�Iaðtþ 1Þ ¼ j;Xðtþ 1Þ ¼ J j IaðtÞ ¼ i

� ¼ dJ ;i;j;

1 � i; j � ma; t � 0; J 2 f0g [ @; ð2:2Þ

where ma is the number of phases of the underlying Markov chain. The constantdJ ;i;j is the probability that a batch J arrives in period tþ 1 and the phase of theunderlying Markov chain is j in period tþ 1, given that the phase was i in period t.Let DJ be an ma �ma matrix with ði; jÞth element dJ ;i;j. Let D0 be an ma �ma

substochastic matrix for no arrival in a period (i.e., XðtÞ ¼ 0). The set of matrices�D0;DJ ; J 2 @

�provides all information about the Markov arrival process with

marked transitions.Let D be the sum of all matrices in the set

�D0;DJ ; J 2 @

�. The matrix D is the

transition probability matrix of the underlying Markov chain IaðtÞ. We assume thatD is irreducible and D 6¼ D0. Let ha be the (unique) invariant probability vector ofthe stochastic matrix D, i.e., haD ¼ ha and hae ¼ 1, where e is a column vector withall elements being one. We note that the irreducibility of D implies that the Perron–Frobenius eigenvalue of D0 (i.e., the eigenvalue with the largest modulus) is less thanone since D 6¼ D0. We refer readers to Gantmacher[7] for more about nonnegativematrices.

In steady state, the total arrival rate of batches of customers is given byl ¼ 1� haD0e. Let lJ ¼ haDJe for J 2 @, i.e., the average arrival rate of batches ofthe type J . The arrival rate of type k customers is given by

lðkÞ ¼XJ2@

NðJ ; kÞlJ ; 1 � k � K; ð2:3Þ

where NðJ ; kÞ is the number of appearances of the integer k in the string J . Formulasfor the arrival rates can be derived by using the classical generating functionapproach (See HE[10]; Neuts[17,19]).

The Queueing Process. After a batch of customers has arrived, all customersjoin a single queue according to the order in the batch. All batches are served by

418 HE

Dow

nloa

ded

by [

Cal

ifor

nia

Poly

Pom

ona

Uni

vers

ity]

at 0

1:14

25

Nov

embe

r 20

14

ORDER REPRINTS

a single server on a FCFS basis. Within each batch, customers are served accordingto their order in the batch. Let qðtÞ be a string of integers for the queue at period t,which is obtained after possible service completion and arrival in period t� 1.Suppose that the services of individual customers in a batch can be distinguished. IfqðtÞ ¼ j1j2 � � � jn, then there are n customers in the system at time t, the customerin service is of type j1, the first customer waiting in queue is of type j2; . . . ; andthe last customer waiting in queue is of type jn. These n customers shall be servedin the order j1; j2; . . . ; jn. If a batch J ¼ h1h2 � � �hk arrives next, the queue becomesj1j2 � � � jnh1h2 � � �hk. If the service of customer j1 is completed next, the queuebecomes j2j3 � � � jn and the server starts serving customer j2. If services are definedfor batches only, then the queueing process can be described in terms of batchesof customers in a similar manner. According to HE[10], the service order of customerswithin a batch can be specified easily by arranging the integers in the correspondingstring. For instance, if type 1 customers have service priority over other customerswithin a batch J , then the integer 1 is placed ahead of others in the stringJ (e.g., J ¼ 1114423). In fact, HE[10] has shown that the string representationprovides great flexibility in modeling with respect to the service order within a batchof customers.

The Service Process. We assume that the service process and the arrival processare independent. The service process (of batches) is governed by a semi-Markovchain with ms states. Let Zn be the phase of the semi-Markov process after the nthtransition (service). Let Xn the string representation of the nth batch in service. Thenwe assume

P�Zn ¼ j; sXn

¼ t jXn ¼ J ; Zn�1 ¼ i� ¼ cJ ;i;jðtÞ;

1 � i; j � ms; n � 1; J 2 @; ð2:4Þ

where sJ is the service time of a batch J and t is a positive integer. We assume that theservice time of any batch is at least one. In Eq. (2.4), the current service is in thephase i and the phase becomes j after the service completion, which is the phaseof the next service. Note that, if the queueing system is empty, the phase of theunderlying semi-Markov chain the phase of the next service. The constant cJ ;i;jðtÞis the probability that the service time is t and the phase becomes j after the comple-tion of the service, given that the current phase is i and the batch in service is of typeJ . Let CJðtÞ be an ms �ms matrix with ði; jÞth element cJ ;i;jðtÞ. The set of matrices�CJðtÞ; t � 1; J 2 @� provides all information about the service process. For any

J 2 @, we denote

C ¼X1t¼1

CJðtÞ; CðtÞ ¼XJ2@

CJðtÞ; t � 1: ð2:5Þ

The matrix C is the transition probability matrix of the underlying Markov chain�Zn; n � 0

�, which is independent of J (i.e., the type of batch in service has no impact

on the environment). We assume that C is irreducible. Let hs be the (unique)

A Discrete Time MMAP[K]/SM[K]/1/FCFS Queue 419

Dow

nloa

ded

by [

Cal

ifor

nia

Poly

Pom

ona

Uni

vers

ity]

at 0

1:14

25

Nov

embe

r 20

14

ORDER REPRINTS

invariant probability vector of the stochastic matrix C, i.e., hsC ¼ hs and hse ¼ 1. Insteady state, the mean service time of a batch J can be calculated as:Ehs;JðsÞ ¼ hs

�P1t¼1 tCJðtÞ

�e. The service rate of batches of the type J is given as,

mJ ¼ Ehs;JðsÞ� ��1

, i.e., the average number of type J batches that can be served perperiod of time.

The traffic intensity of the queueing system is defined as:

r ¼XJ2@

lJ=mJ : ð2:6Þ

According to Loynes[16], the queueing system can reach its steady state if and only ifr < 1. Therefore, throughout this paper, we assume r < 1.

In the above definition of the queueing model, the service process is general.Later in this paper, we shall consider some special cases such as the case with inde-pendent service times (MMAP[K ]=GI[K ]=1) and the case with PH-distributed servicetimes (MMAP[K]=PH[K]=1).

3. THE GENERALIZED TOTAL WORKLOAD PROCESS

The basic idea to analyze the waiting times originates from the following funda-mental relationship for waiting times in queues. Let wn be the (actual) waiting time ofthe nth batch. Then we have

wnþ1 ¼ max�0;wn þ sXn

� tnþ1�; n � 0; ð3:1Þ

where tnþ1 is the length of the time between the arrivals of the nth batch and theðnþ 1Þst batch, Xn is the string representation of the nth batch, and SXn

is the servicetime of the nth batch. The process

�wn; n � 0

�has a repeating structure, which is

difficult to deal with (see Zhao et al.[29]). To study the waiting times, we constructsome processes closely related to

�wn; n � 0

�, which can be dealt with under certain

conditions on the arrival or the service process. Some examples of such processes arethe age process and the total workload (the virtual waiting time) process. The totalworkload process is used in this paper and the age process is used in HE[11] for thestudy of waiting times and sojourn times.

Based on Eq. (3.1), we introduce a process vgðtÞ related to the total workload inthe queueing system as

vgðtÞ ¼ wnðtÞ þ sXnðtÞ � ðt� tnðtÞÞ: ð3:2Þ

where nðtÞ represents the ordinal number of the last batch arrived in or before periodt, and tnðtÞ is the arrival time of the nðtÞth batch. It is readily seen that nðtÞ;wnðtÞ;XnðtÞ,SXnðtÞ , and tnðtÞ update their values if a batch arrives in period t. The relationshipbetween nðtÞ;wnðtÞ; tnðtÞ, SXnðtÞ , and vgðtÞ is shown in Fig. 3.1.

420 HE

Dow

nloa

ded

by [

Cal

ifor

nia

Poly

Pom

ona

Uni

vers

ity]

at 0

1:14

25

Nov

embe

r 20

14

ORDER REPRINTS

The process fvgðtÞ; t � 0g evolves as follows. At a batch arrival epoch (say thenth batch), the total workload in the queueing system is calculated as wn þ SXn

,which is the value of vgðtÞ in that period. During the arrival of the next batch, thevalue of vgðtÞ decreases by one per period. If vgðtÞ � 0; vgðtÞ represents the totalworkload in the queueing system in period t. If vgðtÞ < 0, the queueing system isempty in period t. The value �vgðtÞ indicates how long the current idle period hasbeen (or the age of that idle period in period t). When the next batch arrives, thevalue of vgðtÞ is updated as the total workload in that period and the next cyclebegins. Note that, if vgðtÞ is negative and there is an arrival, then vgðtÞ jumps upwardto the service time of the arrived batch. Compared to the process

�wn; n � 0

�,�

vgðtÞ; t � 0�

provides information about the workload at any time epoch. Infact,

�wn; n � 0

�can be considered as an embedded process of

�vgðtÞ; t � 0

�at

batch arrival epochs.In order to construct a Markov chain related to

�vgðtÞ; t � 0

�, we introduce the

following auxiliary variables. We define a process�IsðtÞ; t � 0

�from the Markov

chain�Zn; n � 0

�as follows: IsðtÞ ¼ Zn if the nth batch is the last one arrived in

or before period t. Thus, IsðtÞ is constant between two consecutive batch arrivals.Recall that XðtÞ is the batch that arrived in period t. If there is no arrival in period t,then XðtÞ ¼ 0. Then Eq. (3.2) leads to

vgðtþ 1Þ ¼ vgðtÞ � 1; if XðtÞ ¼ 0;maxf0; vgðtÞ � 1g þ sXðtÞ; if XðtÞ 6¼ 0:

�ð3:3Þ

Since the arrival process is governed by a Markov chain, the process�ðvgðtÞ; IaðtÞ; IsðtÞÞ; t � 0�

has the Markovian property during no arrival periods.The phase change in the service process is determined by the Markov chain�Zn; n � 0

�. The service time of the batches depends on the Markov chain�

Zn; n � 0�

as well. Thus, the process�ðvgðtÞ; IaðtÞ; IsðtÞÞ; t � 0

�has the

Markovian property in batch arrival periods. Therefore, the process�ðvgðtÞ; IaðtÞ; IsðtÞÞ; t � 0�is a Markov chain. Since the value of vgðtÞ can decrease

at most by one per period, vgðtÞ has the skip-free to the left property. Therefore,�ðvgðtÞ; IaðtÞ; IsðtÞÞ; t � 0�is a Markov chain of M=G=1 type with no boundary at

the level zero. The variables vgðtÞ; IaðtÞ, and IsðtÞ take integer values and their rangesare: �1 < vgðtÞ <1; 1 � IaðtÞ � ma; and 1 � IsðtÞ � ms. A typical sample path ofvgðtÞ is shown in Fig. 3.2.

Figure 3.1. Variables vgðtÞ; nðtÞ; tnðtÞ;wnðtÞ, and SXnðtÞ when vgðtÞ > 0.

A Discrete Time MMAP[K]/SM[K]/1/FCFS Queue 421

Dow

nloa

ded

by [

Cal

ifor

nia

Poly

Pom

ona

Uni

vers

ity]

at 0

1:14

25

Nov

embe

r 20

14

ORDER REPRINTS

The transition probability matrix of�ðvgðtÞ; IaðtÞ; IsðtÞÞ; t � 0

�is given as

Pg ¼

..

.

�1012...

� � � � 1 0 1 2 3 . . .

. .. . .

. ... ..

. ... ..

. � � �. ..

0 0 A1 A2 A3 � � �A0 0 A1 A2 A3 � � �

A0 A1 A2 A3 � � �A0 A1 A2 � � �

. .. . .

. . ..

0BBBBBBBBB@

1CCCCCCCCCA level 0; i:e:; vgðtÞ ¼ 0;

ð3:4Þwhere

A0 ¼ D0 � I; An ¼XJ2@

DJ � CJðnÞ; n � 1; ð3:5Þ

where the notation ‘‘�’’ is for Kronecker product of matrix (see Gantmacher[7]). Notethat the value of vgðtÞ determines the level of the Markov chain. Also note that thePerron–Frobenius eigenvalue of A0 is equal to the Perron–Frobenius eigenvalue ofD0; which is less than one. The transition probabilities in Pg can be verified as follows:

Pfvgðtþ 1Þ ¼ n� 1; Iaðtþ 1Þ ¼ j0; Isðtþ 1Þ ¼ i0 j vgðtÞ ¼ n; IaðtÞ ¼ j; IsðtÞ ¼ ig

¼ ðD0Þj;j0 ; if i ¼ i0;0; if i 6¼ i0.

�ð3:6Þ

Pfvgðtþ 1Þ ¼ n� 1þ s; Iaðtþ 1Þ ¼ j0; Isðtþ 1Þ ¼ i0 j vgðtÞ ¼ n; IaðtÞ ¼ j; IsðtÞ ¼ ig¼XJ2@

PfJðtÞ ¼ J ; Iaðtþ 1Þ ¼ j0 j IaðtÞ ¼ jg

� PfsJ ¼ s; Isðtþ 1Þ ¼ i0 j IsðtÞ ¼ ig

Figure 3.2. A sample path of vgðtÞ.

422 HE

Dow

nloa

ded

by [

Cal

ifor

nia

Poly

Pom

ona

Uni

vers

ity]

at 0

1:14

25

Nov

embe

r 20

14

ORDER REPRINTS

¼XJ2@ðDJÞi;j0 ðCJðsÞÞi;j0 ; s � 1; n � 1;

Pfvgðtþ 1Þ ¼ s; Iaðtþ 1Þ ¼ j0; Isðtþ 1Þ ¼ i0 j vgðtÞ ¼ n; IaðtÞ ¼ j; IsðtÞ ¼ ig¼XJ2@ðDJÞi;j0 ðCJðsÞÞi;j0 ; s � 1; n � 0:

We shall use the steady state distribution of the Markov chain fðvgðtÞ; IaðtÞ;IsðtÞÞ; t � 0g to find the distributions of waiting times. For that purpose, we showthat such a steady state distribution exists if the queueing system is stable, i.e.,r < 1. First, we prove the following properties associated with the transition blocksin Pg. Define

A�ðzÞ ¼X1n¼0

znAn; z � 0; A ¼ A�ð1Þ ¼X1n¼0

An; ð3:7Þ

if the summations are well-defined. By definition (3.5), we have

A ¼ D0 � I þX1n¼1

XJ2@

DJ � CJðnÞ ¼ D0 � I þXJ2@

DJ � C: ð3:8Þ

Remark 3.1. Before proceeding, we like to discuss the irreducibility of the matrixA and the Markov chain fðvgðtÞ; IaðtÞ; IsðtÞÞ; t � 0g, since the steady state analysisof the queueing system depends largely on the assumption that A and fðvgðtÞ;IaðtÞ; IsðtÞÞ; t � 0g are irreducible. Unfortunately, neither A nor fðvgðtÞ; IaðtÞ; IsðtÞÞ;t � 0g is guaranteed to be irreducible under the irreducibility assumptions on thematrices D and C made in Sec. 2. For instance, let

nmax ¼ max�t : CJðtÞ 6¼ 0; for some J 2 @�: ð3:9Þ

It is possible that nmax ¼ 1. If nmax ¼ 1, then these states with vgðtÞ � 2 are notreachable from states with vgðtÞ � 1. Furthermore, it is possible for A andfðvgðtÞ; IaðtÞ; IsðtÞÞ; t � 0g to have several closed classes of states. For example, if

D0 ¼�0:1 00 0

�; D�D0 ¼

�0 0:91 0

�; C ¼

�0 11 0

�; ð3:10Þ

for which the matrix D is irreducible and aperiodic and C is irreducible, the matrix A

has two closed subsets fð1; 1Þ; ð2; 2Þg and fð1; 2Þ; ð2; 1Þg. To analyze the Markovchain, we must identify the closed classes of states and remove transient states. Thenwe concentrate on the irreducible subsets. Therefore, in this paper, we shall assumethat all transient states have been removed and the Markov chain fðvgðtÞ; IaðtÞ;IsðtÞÞ; t � 0g is irreducible and aperiodic.

Denote by wðzÞ the Perron–Frobenius eigenvalue of the nonnegative matrixA�ðzÞ. Let uðzÞ and vðzÞ be the left and right eigenvectors corresponding to wðzÞ,

A Discrete Time MMAP[K]/SM[K]/1/FCFS Queue 423

Dow

nloa

ded

by [

Cal

ifor

nia

Poly

Pom

ona

Uni

vers

ity]

at 0

1:14

25

Nov

embe

r 20

14

ORDER REPRINTS

respectively. The two vectors uðzÞ and vðzÞ are normalized by uðzÞvðzÞ ¼ 1 anduðzÞe ¼ 1. If A is irreducible, then A�ðzÞ is irreducible and all the elements of thevectors uðzÞ and vðzÞ are positive for z > 0. According to Neuts[17], the function wðzÞand the vectors uðzÞ and vðzÞ can be chosen as differentiable functions.

Lemma 3.1. Assume that A is irreducible. The vector ha � hs is the invariantprobability vector of A. At z ¼ 1, we have wð1Þ ¼ 1 and ðha � hsÞSn nAne ¼wð1Þð1Þ ¼ r. Consequently, wð1Þð1Þ < 1 if and only if r < 1. (Note that wð1Þð1Þ is thefirst derivative of the function wðzÞ at z ¼ 1.)

Proof. From the expression of A in Eq. (3.8), it is easy to verify that the vectorha � hs is the invariant probability vector of the stochastic matrix A. It is also easyto see wð1Þ ¼ 1. Furthermore, we have uð1Þ ¼ ha � hs and vð1Þ ¼ e. By uðzÞe ¼ 1,we have uð1ÞðzÞe ¼ 0. By taking derivatives on both sides of uðzÞA�ðzÞ ¼ wðzÞuðzÞ,we obtain uð1ÞðzÞA�ðzÞ þ uðzÞA�ð1ÞðzÞ ¼ wð1ÞðzÞuðzÞ þ wðzÞuð1ÞðzÞ. Letting z ¼ 1 andmultiplying e on both sides of the equation, we obtain uð1ÞA�ð1Þð1Þe ¼ wð1Þð1Þ, i.e.,wð1Þð1Þ ¼ ðha � hsÞ

�P1n¼0 nAn

�e, which can be evaluated as

ðha � hsÞX1n¼0

nAn

!e ¼ ðha � hsÞ

X1n¼1

nXJ2@

DJ � CJðnÞ !

e

¼ ðha � hsÞXJ2@

DJ �X1n¼1

nCJðnÞ ! !

e

¼XJ2@ðhaDJeÞ hs

X1n¼1

nCJðnÞe !

¼XJ2@

lJEh;JðsJÞ ¼XJ2@

lJ=mJ ¼ r: ð3:11Þ

Therefore, wð1Þð1Þ ¼ r. Then wð1Þð1Þ < 1 if and only if r < 1. This completes the proofof Lemma 3.1. &

Denote by G an ðmamsÞ � ðmamsÞ matrix that is the minimal nonnegativesolution to

G ¼X1n¼0

AnGn ¼ D0 � I þ

XJ2@

X1n¼1ðDJ � CJðnÞÞGn

!: ð3:12Þ

We refer to Latouche and Ramaswami[15] and Neuts[17,19] for more about the matrixG. Let g be the (unique) invariant probability vector of G if G is stochastic.

Theorem 3.2. If the Markov chain�ðvgðtÞ; IaðtÞ; IsðtÞÞ; t � 0

�is irreducible, it is

positive recurrent if and only if r < 1.

Proof. It is easy to see from the structure of the transition matrix Pg (see Eq. (3.4))that A must be irreducible if the Markov chain is irreducible. Therefore, by Lemma3.1, the vector ha � hs is the invariant probability vector of A.

424 HE

Dow

nloa

ded

by [

Cal

ifor

nia

Poly

Pom

ona

Uni

vers

ity]

at 0

1:14

25

Nov

embe

r 20

14

ORDER REPRINTS

We use the mean-drift method to prove the theorem. First, we prove thenecessity of the condition r < 1. Note that the level of states of the Markov chain isdetermined by the value of vgðtÞ. We consider the levels with positive vgðtÞ. The tran-sitions among these levels are the same as a classical M=G=1 type Markov chain. Ifthe Markov chain

�ðvgðtÞ; IaðtÞ; IsðtÞÞ; t � 0�is positive recurrent, then the mean first

passage time from any positive level to the level zero must be finite. By Neuts[17], wemust have wð1Þð1Þ < 1. Therefore, we must have r < 1. For completeness, we give thefollowing proof based on the mean drift method.

Let v be the (column vector) mean first passage time from the level n to the leveln� 1, for n � 1. The vector v is independent of the level nð>0Þ because of theM=G=1structure. Since the Markov chain is positive recurrent, every element of v is positiveand finite. By conditioning on the first transition, the following equation can beestablished for v:

v ¼ eþX1n¼1

An

Xn�1i¼0

Giv

!¼ eþ

X1n¼1

An

Xn�1i¼0

Gi

!v: ð3:13Þ

By Eq. (3.13), the Perron–Frobenius eigenvalue of the matrixP1

n¼1 An

Pn�1i¼0 Gi must

be less than one. Since the Markov chain is positive recurrent, G is a stochasticmatrix. It can be shown that I �Gþ eg is invertible. By routine calculations, weobtain

X1n¼1

An

Xn�1i¼0

Gi ¼ ðA�GÞðI �Gþ egÞ�1 þX1n¼1

nAn

!eg: ð3:14Þ

Premultiplying ha � hs on both sides of Eq. (3.14), we obtain

ðha � hsÞX1n¼1

An

Xn�1i¼0

Gi

!¼ ha � hs þ ðha � hsÞ

X1n¼1

nAne� 1

!g: ð3:15Þ

Postmultiplying the right eigenvector corresponding to the Perron–Frobenius eigenvalueofP1

n¼1 An

Pn�1i¼0 Gi on both sides of Eq. (3.15), we find that ðha � hsÞ

Pn nAne < 1,

which implies wð1Þð1Þ < 1. Therefore, by Lemma 3.1, we must have r < 1.

Now, we assume r < 1. By Lemma 3.1, we have wð1Þð1Þ < 1. Thus, we havewðzÞ < z for some z > 1 and close to one. Then we have A�ðzÞvðzÞ ¼ wðzÞvðzÞ <zvðzÞ. Since the Perron–Frobenius eigenvalue of A0 is less than one (because thePerron–Frobenius eigenvalue of D0 is less than one), the matrix I � A0 is invertible.Every elements of the vector ðI � A0Þ�1vðzÞ is finite and positive. With z and vðzÞ,we define a (vector) Lyapunov function (test-function) as follows: for states in thelevel n,

fðnÞ ¼ znvðzÞ; n � 0;

�nðI � A0Þ�1vðzÞ; n < 0.

�ð3:16Þ

(Note: Here we abuse the notation a bit; we use ‘‘n’’ to represent all states in thelevel n.) Since A�ðzÞ is irreducible, every element of the vector vðzÞ is positive.

A Discrete Time MMAP[K]/SM[K]/1/FCFS Queue 425

Dow

nloa

ded

by [

Cal

ifor

nia

Poly

Pom

ona

Uni

vers

ity]

at 0

1:14

25

Nov

embe

r 20

14

ORDER REPRINTS

Therefore, fðnÞ ! 1; jnj ! 1. Let e1 ¼ z� wðzÞð>0Þ and e2 ¼ e1 min�1;

min�ðvðzÞÞi��. We choose n0 large enough so that, for n > n0,

ðA�ðzÞ � A0ÞvðzÞ þ A0ðI � A0Þ�1vðzÞ � nvðzÞ � �e1vðzÞ: ð3:17Þ

The mean-drift to level zero is calculated as follows. For n � 1,

Eðfðagðtþ 1ÞÞ � fðagðtÞÞ j agðtÞ ¼ n; IaðtÞ; IsðtÞÞ¼ zn�1ðA�ðzÞvðzÞ � zvðzÞÞ � �zn�1e1vðzÞ � �e1vðzÞ � �e2e: ð3:18Þ

For �n < �n0, we have

Eðfðagðtþ 1ÞÞ � fðagðtÞÞ j agðtÞ ¼ �n; IaðtÞ; IsðtÞÞ¼ ðA�ðzÞ � A0ÞvðzÞ þ ðnþ 1ÞA0ðI � A0Þ�1vðzÞ � nðI � A0Þ�1vðzÞ¼ ðA�ðzÞ � A0ÞvðzÞ þ A0ðI � A0Þ�1vðzÞ � nvðzÞ� �e1vðzÞ � �e2e: ð3:19Þ

We also have, for �n0 � n � 0,

Eðfðagðtþ 1ÞÞ j agðtÞ ¼ �n; IaðtÞ; IsðtÞÞ¼ ðA�ðzÞ � A0ÞvðzÞ þ ðnþ 1ÞA0ðI � A0Þ�1vðzÞ� A�ðzÞvðzÞ þ ðn0 þ 1ÞA0ðI � A0Þ�1vðzÞ <1: ð3:20Þ

Thus, the mean-drifts away from level zero, with respect to the Lyapunov functiongiven in Eq. (3.16), are all less than �e2, except for a finite number of states. There-fore, the Markov process is positive recurrent by Foster’s criterion (see Theorem2.2.3 in Fayolle et al.[5]). This completes the proof of Theorem 3.2. &

4. STEADY STATE DISTRIBUTIONS

In this section, we consider the stationary distribution of�ðvgðtÞ; IaðtÞ;

IsðtÞÞ; t � 0�

under r < 1 and the irreducibility conditions made in Theorem 3.2.Denote by p ¼ ð. . . ; pð�1Þ; pð0Þ; pð1Þ; . . .Þ the steady state distribution of

�ðvgðtÞ;IaðtÞ; IsðtÞÞ; t � 0

�, i.e., pPg ¼ p; pe ¼ 1, where pðnÞ ¼ ð. . . ; pi;jðnÞ; . . .Þ is a row

vector of the size mams, and

pi;jðnÞ ¼ limt!1PfvgðtÞ ¼ n; IaðtÞ ¼ i; IsðtÞ ¼ j j vgð0Þ; Iað0Þ; Isð0Þg: ð4:1Þ

We further assume that the Markov chain�ðvgðtÞ; IaðtÞ; IsðtÞÞ; t � 0

�is

aperiodic so that the steady state distribution (invariant probability measure) p isunique. First, we find

�pð0Þ; pð�1Þ; pð�2Þ; . . .� in terms of pð0Þ. By pPg ¼ p, we

have pð�n� 1Þ ¼ pð�nÞA0, for n � 0, which leads to

pð�nÞ ¼ pð0ÞðA0Þn for n � 0: ð4:2Þ

426 HE

Dow

nloa

ded

by [

Cal

ifor

nia

Poly

Pom

ona

Uni

vers

ity]

at 0

1:14

25

Nov

embe

r 20

14

ORDER REPRINTS

Thus, to find p, we only need to concentrate on�pð0Þ; pð1Þ; pð2Þ; . . .�. In fact, we can

construct another Markov chain, which is related to the (actual) workload process,to find

�pð0Þ; pð1Þ; pð2Þ; . . .�. The construction of the Markov chain is given as

follows. By pPg ¼ p, we have

pðnÞ ¼ pðnþ 1ÞA0 þ pðnÞA1 þ � � � þ pð1ÞAn þX1i¼0

pð�iÞ !

An; n � 1: ð4:3Þ

Denote by pð�1; 0� ¼P1i¼0 pð�iÞ ¼ pð0ÞðI � A0Þ�1. Note that the matrixðI � A0Þ�1 is finite and nonnegative. Then the probability vector ðpð�1; 0�;pð1Þ; pð2Þ; . . .Þ is the steady state distribution of the following transition probabilitymatrix

Pv ¼

A0 A1 A2 A3 . . .A0 A1 A2 A3 . . .

A0 A1 A2. ..

. .. . .

. . ..

0BBB@

1CCCA: ð4:4Þ

For the Markov chain�ðvgðtÞ; IaðtÞ; IsðtÞÞ; t � 0

�, if we only observe the process

when vgðtÞ � 0, we obtain a process�ðvðtÞ; IaðtÞ; IsðtÞÞ; t � 0

�, where vðtÞ is the

(actual) total workload in the system. It is easy to see that�ðvðtÞ; IaðtÞ; IsðtÞÞ; t � 0�is an M=G=1 type Markov chain and its transition prob-

ability matrix is given by Eq. (4.4). Note that, to obtain the steady state waiting timedistributions, we can directly consider the Markov chain

�ðvðtÞ; IaðtÞ; IsðtÞÞ; t � 0�.

By considering the Markov chain�ðvgðtÞ; IaðtÞ; IsðtÞÞ; t � 0

�, however, we get more

information about the queueing process, especially about the idle periods.Denote by p�ðzÞ ¼ pð�1; 0� þP1n¼1 znpðnÞ. From Eq. (4.4), it is easy to obtain

the following equation for p�ðzÞ:

p�ðzÞðzI � A�ðzÞÞ ¼ pð�1; 0�ðz� 1ÞA�ðzÞ: ð4:5Þ

Theorem 4.1. If the Markov chain�ðvgðtÞ; IaðtÞ; IsðtÞÞ; t � 0

�is irreducible and

aperiodic and r < 1, then p�ð1Þ ¼ ha � hs and pð�1; 0� ¼ ð1� rÞg, pð0Þ ¼ð1� rÞgðI � A0Þ, where g is the invariant probability vector of G.

Proof. Letting z go to 1 on both sides of Eq. (4.5), we obtain p�ð1ÞðI � AÞ ¼ 0.Since p�ð1Þ is a probability vector, it must be the invariant probability vector of A.By Lemma 3.1, p�ð1Þ ¼ ha � hs.

Dividing both sides of Eq. (4.5) by z� 1 and multiplying both sides by e,yields

pð�1; 0�A�ðzÞe ¼ p�ðzÞ ðzI � A�ðzÞÞez� 1

: ð4:6Þ

Letting z go to 1 on both sides of Eq. (4.6) and using 1’Hopital’s Rule, by

A Discrete Time MMAP[K]/SM[K]/1/FCFS Queue 427

Dow

nloa

ded

by [

Cal

ifor

nia

Poly

Pom

ona

Uni

vers

ity]

at 0

1:14

25

Nov

embe

r 20

14

ORDER REPRINTS

Lemma 3.1, we obtain

pð�1; 0�e ¼ limz!1

p�ðzÞ ðzI � A�ðzÞÞez� 1

¼ p�ð1Þ�1�

X1n¼0

nAn

�e

¼ ðha � hsÞ�1�

X1n¼0

nAn

�e ¼ 1� wfð1Þgð1Þ ¼ 1� r: ð4:7Þ

Consider the embedded Markov chain P0 at the epochs the Markov chain�ðvgðtÞ; IaðtÞ; IsðtÞÞ; t � 0�

enters the level zero. Then we must have pð�1; 0� ¼pð�1; 0�P0. By conditioning on the first transition from the level zero, we obtain

P0 ¼X1n¼0

AnGn ¼ G: ð4:8Þ

Therefore, we must have pð�1; 0� ¼ cg, where c is a constant. Sincepð�1; 0�e ¼ 1� r, then c ¼ 1� r. This completes the proof of Theorem 4.1. &

Once pð�1; 0� is found, other vectors fpð1Þ; pð2Þ; . . .g can be computed by astable recursion developed in Ramaswami[20]: for n � 1,

pðnÞ ¼�pð�1; 0�An þ

Xn�1j¼1

pðjÞAn�jþ1

�ðI � A1Þ�1; ð4:9Þ

where An ¼P1

j¼n AjGj�n for n � 1.

Remark 4.1. The results obtained in Theorem 4.1 and the above algorithm may bevalid for the more general case. For instance, if nmax ¼ 1 (defined in Eq. (3.9)), theMarkov chain

�ðvgðtÞ; IaðtÞ; IsðtÞÞ; t � 0�

is reducible. However, if pð0Þ can beobtained, then

�pð1Þ; pð2Þ; . . .� can be computed by using Eq. (4.9). For this case,

An ¼ 0 for n � 2. Equation (4.9) gives pðnÞ ¼ 0 for n � 2.

Now, we return to the total workload and the waiting time of an arbitrary batch.Let ng be the generic random variable for the total workload in the system in steadystate, w be the generic random variable for the waiting time of an arbitrary batch(i.e., the waiting time of the first customer in the batch), wJ be the generic randomvariable for the waiting time of an arbitrary batch J , d be the generic random vari-able for the sojourn time of an arbitrary batch (the total time the whole batch is inthe system), and dJ be the generic random variable for the sojourn time of an arbi-trary batch J . We also denote by Ia the generic random variable of the underlyingMarkov chain of the arrival process in steady state and by Is the generic randomvariable of the service process in steady state.

Theorem 4.2. If the queueing system is stable and the Markov chain�ðvgðtÞ; IaðtÞ; IsðtÞÞ; t � 0�is irreducible and aperiodic, the distributions of the total

428 HE

Dow

nloa

ded

by [

Cal

ifor

nia

Poly

Pom

ona

Uni

vers

ity]

at 0

1:14

25

Nov

embe

r 20

14

ORDER REPRINTS

workload, waiting times, and sojourn times are given by:

Pfvg ¼ ng ¼ pðnÞe; �1 < n <1: ð4:10Þ

Pfw ¼ ng ¼1

l

�ð1� rÞgððD�D0Þ � IÞeþ pð1ÞððD�D0Þ � IÞe�; n ¼ 0;

1

lpðnþ 1ÞððD�D0Þ � IÞe; n � 1:

8>><>>:

ð4:11Þ

PfwJ ¼ ng ¼1

lJ

�ð1� rÞgðDJ � IÞeþ pð1ÞðDJ � IÞe�; n ¼ 0;

1

lJpðnþ 1ÞðDJ � IÞe; n � 1:

8>><>>: ð4:12Þ

Pfd ¼ ng ¼ 1

l

�ð1� rÞgððD�D0Þ � CðnÞÞ

þXnt¼1

pðtÞððD�D0Þ � Cðnþ 1� tÞÞ�e; n � 1: ð4:13Þ

PfdJ ¼ ng ¼ 1

lJ

�ð1� rÞgðDJ � CJðnÞÞ

þXnt¼1

pðtÞðDJ � CJðnþ 1� tÞÞ�e; n � 1: ð4:14Þ

Proof. Equation (4.10) is obvious. Recall that, if vg < 0, the queueing system isempty in that period and the system has been idle for �vg periods of time.

Equation (3.2) shows that the waiting time of a batch is the (generalized) totalworkload right before its arrival. Therefore, the distribution of waiting time wJ ofan arbitrary batch of type J can be obtained by conditioning on the arrival of a batchJ at an arbitrary time, which can be obtained as follows, for n � 1,

PfwJ ¼ ng ¼ Pfvg ¼ nþ 1 jBatch J arrives nextg

¼ Pfvg ¼ nþ 1;Batch J arrives nextgPfBatch J arrives nextg

¼ 1

lJ

Xma

i¼1

Xms

j¼1Pfvg ¼ nþ 1; Ia ¼ i; Is ¼ j;Batch J arrives nextg

¼ 1

lJ

Xma

i¼1

Xms

j¼1Pfvg ¼ nþ 1; Ia ¼ i; Is ¼ jg

� PfBatch J arrives nextg j vg ¼ nþ 1; Ia ¼ i; Is ¼ jg

A Discrete Time MMAP[K]/SM[K]/1/FCFS Queue 429

Dow

nloa

ded

by [

Cal

ifor

nia

Poly

Pom

ona

Uni

vers

ity]

at 0

1:14

25

Nov

embe

r 20

14

ORDER REPRINTS

¼ 1

lJ

Xma

i¼1

Xms

j¼1Pfvg ¼ nþ 1; Ia ¼ i; Is ¼ jg

� PfBatch J arrives next j Ia ¼ ig

¼ 1

lJ

Xma

i¼1

Xms

j¼1pi;jðnþ 1ÞðDJeÞi; ð4:15Þ

which leads to the second expression in Eq. (4.12). If n ¼ 0, we have fwJ ¼ 0g ¼�vg � 1jBatch J batch arrives next

�, which leads to the first expression in Eq. (4.12).

The distribution of w can be obtained similarly. The distributions of the sojourntimes are obtained as the convolution of the distributions of waiting time and servicetime by taking the phase of the service process into consideration. This completes theproof of Theorem 4.2.

Note that the computation of the moments of the waiting times can be done byusing existing formulas for the M=G=1 type Markov chains (see Neuts[17,19]). Detailsare omitted. The computation of the waiting time distribution of an arbitrary type kcustomer can be a little bit involved, since the service times of customers in a batchmay not be independent. In other words, if the service time is defined at the batchlevel, the waiting time distribution of a type k customer cannot be found explicitly.Nonetheless, if more information on the service process is available, it might bepossible to find the distribution of waiting times of individual types of customers.An example is given in Sec. 5.

To end this section, we point out that there is a close relation between the ageprocess of the batch in service and the total workload process. We define the ageof a batch in period t as the total time the batch has been in the queueing system,given that the batch is in the system in period t. The generalized age processfagðtÞ; t � 0g of the batch in service or to be served next (if the system is empty)is defined as

agðtÞ ¼ wnðtÞ þ sXnðtÞ � tnðtÞþ1 þ t� tnðtÞ; ð4:16Þ

where nðtÞ is the ordinal number of the last batch served in or before period t andtnðtÞ is the departure period of the nðtÞth batch. The values of nðtÞ, wnðtÞ, and tnðtÞare updated if a departure occurs in period t, where wnðtÞ can be computed by usingEq. (3.1). According to Corollary 4.2 in HE[11], agðtÞ and vgðtÞ have the samesteady-state distribution. Thus, Eq. (4.10) can be used for computing the steady statedistribution of the age of the batch in service at an arbitrary time as well.

5. THE DISCRETE TIME MMAP[K ]=GI [K ]=1 QUEUE

For the discrete time MMAP[K ]=GI[K ]=1 queue, the service times of individualcustomers are independent with general distributions, i.e., ms ¼ 1. Consequently, theservice times of batches are independent as well. Denote by sk the service time of a

430 HE

Dow

nloa

ded

by [

Cal

ifor

nia

Poly

Pom

ona

Uni

vers

ity]

at 0

1:14

25

Nov

embe

r 20

14

ORDER REPRINTS

type k customer. Then the service rate of type k customers is given bymðkÞ ¼ 1=ðEðskÞÞ. The service time of a batch J ¼ j1j2 � � � jn is sJ ¼

Pni¼1 sji with mean

EðsjÞ ¼Pn

i¼1 EðsjiÞ. By definition (2.6), we have

r ¼XJ2@

lJ=mJ ¼XJ2@

lJXjJji¼1

EðsjiÞ ¼XJ2@

lJXKk¼1

NðJ ; kÞEðskÞ

¼XKk¼1

XJ2@

lJNðJ ; kÞEðskÞ ¼XKk¼1

lðkÞ=mðkÞ: ð5:1Þ

Thus, for the MMAP[K ]=GI [K ]=1 queue, definition (2.6) for the traffic intensity isconsistent with the classical definition.

The distribution of the total workload and the waiting time of batches can becalculated by using formulas (4.10) and (4.11). For this special case, we are able tocompute the waiting times of individual types of customers. Let wðkÞ be the genericrandom variable for the waiting time of an arbitrary type k customer. Based onEq. (4.11), we have, for n � 0 and 1 � k � K,

PfwðkÞ ¼ ng

¼ � 1

lðkÞ

�ð1� rÞgþ pð1Þ�XJ2@

XjJjt¼1ðDJ � IÞe Pfsj1 þ sj2 þ � � � þ sjt�1 ¼ ngdfjt¼kg

þ 1

lðkÞ

Xnl¼2

XJ2@

XjJjt¼1

pðlÞðDJ � IÞe Pfsj1 þ sj2 þ � � � þ sjt�1 ¼ nþ 1� lgdfjt¼kg

¼XJ2@

lJlðkÞ

XjJ jt¼1

PfwJ þ sj1 þ sj2 þ � � � þ sjt�1 ¼ ngdfjt¼kg; ð5:2Þ

where df:g is the indicator function. Let dðkÞ be the generic random variable for thesojourn time of an arbitrary type k customer. Note that, for the definition of dðkÞ, weassume that a customer leaves the system immediately after its service. We have, forn � 1 and 1 � k � K,

PfdðkÞ ¼ ng ¼XJ2@

lJlðkÞ

XjJ jt¼1

PfwJ þ sj1 þ sj2 þ � � � þ sjt ¼ ngdfjt¼kg: ð5:3Þ

An important special case is the discrete time MMAP[K ]=PH[K ]=1 queue, wherethe service times of individual customers are independent and have PH-distributionswith matrix representations fðmk; ak;TkÞ; 1 � k � Kg, where mk is the number ofphases of the PH-distribution, ak is the initial probability vector, and Tk is a substo-chastic matrix. We assume that the service time of any customer is at least one(i.e., ake ¼ 1; 1 � k � K). See Neuts[17] for more about PH-distribution. Denote byT0k ¼ ðI � TkÞe. Since PH-distributions are closed under convolution, the service time

of a batch J also has a discrete time PH-distribution with a matrix representation

A Discrete Time MMAP[K]/SM[K]/1/FCFS Queue 431

Dow

nloa

ded

by [

Cal

ifor

nia

Poly

Pom

ona

Uni

vers

ity]

at 0

1:14

25

Nov

embe

r 20

14

ORDER REPRINTS

�mJ ; aJ ;TJ

�, where, for J ¼ j1j2 � � � jn,

mJ ¼Xni¼1

mji ;

aJ ¼ ðaj1 ; 0; . . . ; 0Þ;

TJ ¼

Tj1 T0j1aj2

Tj2 T0j2aj3

. .. . .

.

Tjn�1 T0jn�1ajn

Tjn

0BBBBBBBBB@

1CCCCCCCCCA; T0

J ¼

0

..

.

..

.

0

T0jn

0BBBBBBBBB@

1CCCCCCCCCA:

ð5:4Þ

According to Neuts[17], the distributions of service times are given by

CJðtÞ ¼ aJTt�1J T0

J ; t � 1; J 2 @: ð5:5ÞThe matrix blocks of the transition matrix of

�ðvgðtÞ; IaðtÞ; IsðtÞÞ; t � 0�

(seeEq. (3.5)) can be computed easily by

A0 ¼ D0; An ¼XJ2@

aJTn�1J T0

J

� �DJ ; n � 1: ð5:6Þ

The steady state distributions of waiting times can be computed accordingly byusing formulas provided in Secs. 3 and 4. Note that for this case, A ¼ D. So A isirreducible if and only if D is irreducible.

We point out that for the discrete time MMAP[K ]=PH[K ]=1 queue, since boththe arrival process and the service process are governed by Markov chains, a QBDprocess can be introduced to describe the total workload process and the age processof the batch in service. That approach was used in HE[11] and Van Houdt andBlondia[28]. The advantage of the QBD approach is that the steady state distributionof the QBD process has a matrix geometric distribution, which is easy to compute.On the other hand, as was pointed out by Van Houdt and Blondia[28], the dimensionof the matrix blocks involved can be quite large for the QBD approach, which is

ma

�1þ

XJ2@

XjJ ji¼1

mji

�: ð5:7Þ

That number can be significantly larger than ma – the dimension of the blocks for theM=G=1 type Markov chain developed in this paper. Even if we consider the compu-tation of PH-distributions used in constructing matrices fAn; n � 0g (see Eq. (5.6)),the largest dimension of matrices involved with the M=G=1 type Markov chain ismaxfma;mJ ; J 2 @g, which can be significantly smaller than the number inEq. (5.7). Therefore, the M=G=1 approach can play an important role in the analysisof the waiting times and sojourn times for the MMAP[K ]=PH[K ]=1 case. Morediscussion on this issue is given in Sec. 6.

432 HE

Dow

nloa

ded

by [

Cal

ifor

nia

Poly

Pom

ona

Uni

vers

ity]

at 0

1:14

25

Nov

embe

r 20

14

ORDER REPRINTS

6. NUMERICAL EXAMPLES

In this section, we outline a scheme for computing distributions of waiting timesand sojourn times and briefly discuss three numerical examples. First, we summarizethe steps for computing performance measures in the following examples.

(1) Compute vectors ha and hs, the arrival rates flJ ; J 2 @g, arrival ratesfmJ ; J 2 @g, and r (by Eq. (2.6)).

(2) Compute transition blocks fA0;A1;A2; . . .g by Eq. (3.5).(3) Compute the matrix G by Eq. (3.12) and the vector g.(4) Compute the vector pð�1; 0� and pð0Þ by Theorem 4.1.(5) Compute vectors fpð0Þ; pð�1Þ; pð�2Þ; . . .g by Eq. (4.2).(6) Compute vectors fpð1Þ; pð2Þ; . . .g by the recursion given in Eq. (4.9).(7) Determine distributions of the total workload, waiting times, and sojourn

times by formulas given in Corollary 4.2 or Eqs. (5.2) and (5.3).

Most of the above steps can be done in a straightforward manner except thecomputation of the matrix G and vectors fpð1Þ; pð2Þ; . . .g. Nonetheless, for the com-putation of G and fpð1Þ; pð2Þ; . . .g, efficient algorithms are available in the literature(Neuts[19]; Ramaswami[20]).

Example 6.1. Consider an MMAP[2]=PH[2]=1 queue with following systemparameters:

K ¼ 2; ma ¼ 2; D0 ¼0:6 0:0

0:4 0:1

� �; D1 ¼

0:0 0:1

0:0 0:0

� �;

D2 ¼0:3 0:0

0:4 0:1

� �;

m1 ¼ 1; a1 ¼ ð1Þ; T1 ¼ ð0:4Þ; m2 ¼ 2;

a2 ¼ ð0:1; 0:9Þ; T2 ¼0:1 0:1

0:1 0:0

� �:

ð6:1Þ

This queueing system has a light traffic with r ¼ 0:5139. For this queueingsystem, a type 1 customer is mostly likely to be followed by type 2 customers.Since type 1 customers have a slower service rate, it is expected that type 2customers face longer waiting time than type 1 customers, which is shown inTable 1. The distributions of waiting times and sojourn times are given inTable 1.

It is interesting to see that Pfv ¼ 0g ¼ 0:4861 ¼ 1� r, which is much smallerthan Pfw ¼ 0g ¼ 0:8125. In fact, for many numerical examples, Pfv ¼ 0gþPfv ¼ 1g Pfw ¼ 0g. The reason is that, for discrete queues, an arrival and adeparture can occur simultaneously. Therefore, if the total workload is one, thewaiting time of the next arrival is actually zero. Next, we examine a queueing systemin heavy traffic.

A Discrete Time MMAP[K]/SM[K]/1/FCFS Queue 433

Dow

nloa

ded

by [

Cal

ifor

nia

Poly

Pom

ona

Uni

vers

ity]

at 0

1:14

25

Nov

embe

r 20

14

ORDER REPRINTS

Example 6.2. Consider an MMAP[5]=PH[5]=1 queue with following systemparameters: K ¼ 5;ma ¼ 3,

D0 ¼0:1 0:1 0

0:2 0:3 0:1

0:1 0 0:3

0B@

1CA; D1 ¼

0:1 0:4 0

0:1 0:2 0

0 0 0

0B@

1CA; D2 ¼

0 0:1 0

0 0:1 0

0 0 0

0B@

1CA;

D3 ¼0:1 0 0

0 0 0

0 0 0:2

0B@

1CA; D4 ¼

0:1 0 0

0 0 0

0 0 0:1

0B@

1CA; D5 ¼

0 0 0

0 0 0

0 0 0:3

0B@

1CA;

ð6:2Þ

mi ¼ 1; ai ¼ ð1Þ; Ti ¼ ð0:1Þ; i ¼ 1; 2;

m3 ¼ 2; a3 ¼ ð0:1; 0:9Þ; T3 ¼0:7 0:1

0:4 0

� �;

mi ¼ 2; ai ¼ ð0; 1Þ; Ti ¼0:7 0:1

0:2 0

� �; i ¼ 4; 5:

ð6:3Þ

The traffic intensity of this queueing system is r ¼ 0:9304. The distributions ofwaiting times are given in Table 2.

Some observations can be made from Table 2. First, compared to Example 6.1,the distributions have a heavier tail. That is, the waiting times are much longer than

Table 1. Distributions of waiting times and sojourn times.

n 0 1 2 3 4 5 6 7 . . .

Pfv ¼ ng 0.4861 0.3340 0.1025 0.0431 0.0186 0.0083 0.0038 0.0017 . . .Pfw ¼ ng 0.8125 0.1063 0.0451 0.0196 0.0088 0.0040 0.0018 0.0008 . . .Pfw1 ¼ ng 0.8557 0.0849 0.0338 0.0140 0.0062 0.0028 0.0012 0.0006 . . .Pfw2 ¼ ng 0.8006 0.1122 0.0482 0.0211 0.0095 0.0043 0.0020 0.0009 . . .Pfd ¼ ng 0 0.6695 0.1895 0.0792 0.0338 0.0151 0.0068 0.0031 . . .Pfd1 ¼ ng 0 0.5134 0.2563 0.1228 0.0575 0.0267 0.0123 0.0057 . . .Pfd2 ¼ ng 0 0.7125 0.1711 0.0671 0.0272 0.0119 0.0053 0.0024 . . .

Table 2. Distributions of waiting times.

n 0 1 2 3 4 5 6 7 . . .

Pfw ¼ ng 0.1585 0.0348 0.0286 0.0263 0.0250 0.0240 0.0231 0.0222 . . .Pfw1 ¼ ng 0.1852 0.0404 0.0294 0.0262 0.0246 0.0235 0.0224 0.0215 . . .Pfw2 ¼ ng 0.1864 0.0414 0.0294 0.0261 0.0245 0.0234 0.0224 0.0214 . . .Pfw3 ¼ ng 0.1333 0.0293 0.0278 0.0264 0.0254 0.0245 0.0237 0.0229 . . .Pfw4 ¼ ng 0.1427 0.0308 0.0280 0.0264 0.0253 0.0244 0.0235 0.0227 . . .Pfw5 ¼ ng 0.1177 0.0268 0.0273 0.0263 0.0255 0.0247 0.0240 0.0232 . . .

434 HE

Dow

nloa

ded

by [

Cal

ifor

nia

Poly

Pom

ona

Uni

vers

ity]

at 0

1:14

25

Nov

embe

r 20

14

ORDER REPRINTS

that of Example 6.1. Second, the waiting times of the five types of customers are sig-nificantly different. For example, Pfw5 ¼ 0g ¼ 0:1177, which is significantly smallerthan Pfw1 ¼ 0g ¼ 0:1852. Thus, in a queueing system, the waiting times of differenttypes of customers can be dramatically different. Similar conclusions can be drawnfor the sojourn times. For this example, the size of the matrix blocks in computationis 3. If the QBD approach or the GI=M=1 approach (HE[11]; Van Houdt andBlondia[28]) is used, the size of the matrix blocks in computation is 27 or 24, whichis significantly larger.

Example 6.3. Consider an MMAP[2]=PH[2]=1 queue with three types of batchesJ1 ¼ 1, J2 ¼ 2, and J3 ¼ 12 and following system parameters: K ¼ 2, ma ¼ 2,

D0 ¼0:5 0

0:2 0:5

� �; D1 ¼

0 0:3

0 0:1

!; D2 ¼

0:2 0

0 0

!;

D12 ¼0 0

0 0:2

� �;

m1 ¼ 1; a1 ¼ ð1Þ; T1 ¼ ð0:4Þ; m2 ¼ 2;

a2 ¼ ð0:9; 0:1Þ; T2 ¼0:1 0

0:1 0:7

� �:

ð6:4Þ

The traffic intensity of this queueing system is r ¼ 0:774. By using Eq. (5.2), thedistributions of waiting times of the two types of customers can be computed andthe results are given in Table 3.

Note that the wJ is the waiting time of an arbitrary batch J and wðiÞ is the waitingtime of an arbitrary type i customer, i ¼ 1; 2. Table 3 shows that the waiting timesof type 2 customers are significantly longer than that of type 1 customers, becausemany type 2 customers are served after a type 1 customer. Thus, the order of servicein a batch can have significant effect on the queueing processes of different types ofcustomers.

ACKNOWLEDGMENTS

The author would like to thank K-C Wang Foundation and Chinese Academyof Science for their support on this research project. The author would like to thank

Table 3. Distributions of waiting times

n 0 1 2 3 4 5 6 7 . . .

Pfw1 ¼ ng 0.4422 0.1051 0.0804 0.0643 0.0522 0.0428 0.0352 0.0292 . . .Pfw2 ¼ ng 0.5176 0.0915 0.0682 0.0548 0.0448 0.0369 0.0306 0.0254 . . .Pfw12 ¼ ng 0.2914 0.1322 0.1048 0.0833 0.0670 0.0544 0.0446 0.0367 . . .Pfwð1Þ ¼ ng 0.3819 0.1159 0.0902 0.0719 0.0581 0.0474 0.0390 0.0322 . . .Pfwð2Þ ¼ ng 0.2070 0.1415 0.1168 0.0955 0.0773 0.0626 0.0510 0.0417 . . .

A Discrete Time MMAP[K]/SM[K]/1/FCFS Queue 435

Dow

nloa

ded

by [

Cal

ifor

nia

Poly

Pom

ona

Uni

vers

ity]

at 0

1:14

25

Nov

embe

r 20

14

ORDER REPRINTS

Dr. Blake for proofreading the paper. The author would also like to thank twoanonymous referees for their valuable comments and suggestions. This researchwas partially supported by a NSERC research grant.

REFERENCES

1. Alfa, S.A. Discrete time queues and matrix-analytic methods. TOP 2002, 10(2), 147–210.

2. Asmussen, S.; Koole, G. Marked point processes as limits of Markovian arrivalstreams. J. Appl. Probab. 1993, 30, 365–372.

3. Cohen, J.W. The Single Server Queue; North-Holland: Amsterdam, 1982.4. Cortizo, D.V.; Garcia, J.; Blondia, C.; Van Houdt, B. FIFO by sets ALOHA

(FS-ALOHA): A collision resolution algorithm for the contention channel inwireless ATM systems. Perform. Eval. 1999, 36–37, 401–427.

5. Fayolle, G.; Malyshev, V.A.; Menshikov, M.V. Topics in the ConstructiveTheory of Countable Markov Chains; Cambridge University Press, 1995.

6. Gail, H.R.; Hantler, S.L.; Taylor, B.A. Non-skip-free M=G=1 and G=M=1 typeMarkov chains. Adv. Appl. Probab. 1997, 29, 733–758.

7. Gantmacher, F.R. The theory of Matrices; Chelsea: New York, 1959.8. Grassmann, W.K.; Jain, J.L. Numerical solutions of waiting time distribution

and idle time distribution of the arithmetic GI=G=1 queue. Oper. Res. 1989,37, 141–150.

9. HE, Qi-Ming. Queues with marked customers. Adv. Appl. Prob. 1996, 28,567–587.

10. HE, Qi-Ming. The versatility of MMAP[K] and the MMAP[K]=G[K]=1 queue.Queueing Syst. 2001, 38 (4), 397–418.

11. HE, Qi-Ming. Age process, total workload, sojourn times, and waiting times ina discrete time SM[K]=PH[K]=1=FCFS queue (submitted).

12. HE, Qi-Ming. Age process, sojourn times, waiting times, and queue lengths in acontinuous time SM[K]=PH[K]=1=FCFS queue (submitted).

13. HE, Qi-Ming; Alfa, A.S. The MMAP[K]=PH[K]=1 queue with a last-come-first-served preemptive service discipline. Queueing syst. 1998, 28, 269–291.

14. HE, Qi-Ming; Neuts, M.F. Markov chains with marked transitions. StochasticProces. Appl. 1998, 74 (1), 37–52.

15. Latouche, G.; Ramaswami, V. Introduction to Matrix Analytic Methods inStochastic Modelling; ASA & SLAM: Philadelphia, USA, 1999.

16. Loynes, R.M. The stability of a queue with non-independent interarrival andservice times. Proc. Cambridge Philos. Soc. 1962, 58, 497–520.

17. Neuts, M.F. Matrix-Geometric Solutions in Stochastic Models: An AlgorithmicApproach; The Johns Hopkins University Press: Baltimore, 1981.

18. Neuts, M.F. Generalizations of the Pollaczek–Khinchin integral method in thetheory of queues. Adv. Appl. Prob. 1986, 18, 952–990.

19. Neuts, M.F. Structured Stochastic Matrices of M=G=1 type and TheirApplications; Marcel Dekker: New York, 1989.

20. Ramaswami, V. Stable recursion for the steady state vector in Markov chainsof M=G=1 type. Stochastic Models 1988, 4, 183–188.

436 HE

Dow

nloa

ded

by [

Cal

ifor

nia

Poly

Pom

ona

Uni

vers

ity]

at 0

1:14

25

Nov

embe

r 20

14

ORDER REPRINTS

21. Sengupta, B. Markov processes whose steady state distribution is matrix-exponential with an application to the GI=PH=1 queue. Adv. Appl. Prob.1989, 21, 159–180.

22. Takine, T. A continuous version of matrix-analytic methods with the skip-freeto the left property. Stochastic Models 1996, 12 (4), 673–682.

23. Takine, T. Queue length distribution in a FIFO single-server queue withultiple arrival streams having different service time distributions. QueueingSyst. 2001, 39, 349–375.

24. Takine, T. A recent progress in algorithmic analysis of FIFO queues withMarkovian arrival streams. J. Korean Math. Soc. 2001, 38 (4), 807–842.

25. Takine, T.; Hasegawa, T. The workload in a MAP=G=1 queue with state-dependent services: Its applications to a queue with preemptive resume priority.Stochastic Models 1994, 10 (1), 183–204.

26. Van Houdt, B.; Blondia, C. The delay distribution of a type k customer in aFCFS MMAP[K]=PH[K]=1 queue. J. Appl. Probab. 2002, 39 (1), 213–222.

27. Van Houdt, B.; Blondia, C. The Waiting Time Distribution of a Type k Custo-mer in a FCFS MMAP[K]=PH[K]=2 Queue, Technical Report.

28. Van Houdt, B.; Blondia, C. The waiting time distribution of a type k customerin a discrete time MMAP[K]=PH[K]=c ðc ¼ 1; 2Þ queue using QBDs. StochasticModels 2004, 20 (1), 55–69.

29. Zhao, Y.Q.; Li, W.; Braun, W.J. Censoring, Factorization, and SpectralAnalysis for Transition Matrices with Block-Repeating Entries, TechnicalReport (No. 355); Laboratory for Research in Statistics and Probability,Carleton University and University of Ottawa, 2001.

30. Yang, T.; Chaudhry, M. On the steady-state queue size distributions ofdiscrete-time GI=G=1 queue. Adv. Appl. Probab. 1996, 28, 1177–1200.

Received May 2003Accepted May 2004

A Discrete Time MMAP[K]/SM[K]/1/FCFS Queue 437

Dow

nloa

ded

by [

Cal

ifor

nia

Poly

Pom

ona

Uni

vers

ity]

at 0

1:14

25

Nov

embe

r 20

14

Request Permission/Order Reprints

Reprints of this article can also be ordered at

http://www.dekker.com/servlet/product/DOI/101081STM200033099

Request Permission or Order Reprints Instantly!

Interested in copying and sharing this article? In most cases, U.S. Copyright Law requires that you get permission from the article’s rightsholder before using copyrighted content.

All information and materials found in this article, including but not limited to text, trademarks, patents, logos, graphics and images (the "Materials"), are the copyrighted works and other forms of intellectual property of Marcel Dekker, Inc., or its licensors. All rights not expressly granted are reserved.

Get permission to lawfully reproduce and distribute the Materials or order reprints quickly and painlessly. Simply click on the "Request Permission/ Order Reprints" link below and follow the instructions. Visit the U.S. Copyright Office for information on Fair Use limitations of U.S. copyright law. Please refer to The Association of American Publishers’ (AAP) website for guidelines on Fair Use in the Classroom.

The Materials are for your personal use only and cannot be reformatted, reposted, resold or distributed by electronic means or otherwise without permission from Marcel Dekker, Inc. Marcel Dekker, Inc. grants you the limited right to display the Materials only on your personal computer or personal wireless device, and to copy and download single copies of such Materials provided that any copyright, trademark or other notice appearing on such Materials is also retained by, displayed, copied or downloaded as part of the Materials and is not removed or obscured, and provided you do not edit, modify, alter or enhance the Materials. Please refer to our Website User Agreement for more details.

Dow

nloa

ded

by [

Cal

ifor

nia

Poly

Pom

ona

Uni

vers

ity]

at 0

1:14

25

Nov

embe

r 20

14