morfismos, vol 7, no 1, 2003

VOLUMEN 7NÚMERO 1

ENERO A JUNIO DE 2003 ISSN: 1870-6525

MORFISMOSComunicaciones EstudiantilesDepartamento de Matematicas

Cinvestav

Editores Responsables

• Isidoro Gitler • Jesus Gonzalez

Consejo Editorial

• Felipe Gayosso • Samuel Gitler• Onesimo Hernandez-Lerma • Raul Quiroga Barranco

• Enrique Ramırez de Arellano • Francisco Ramırez Reyes• Jose Rosales Ortega • Mario Villalobos Arias

• Heraclio Villarreal Rodrıguez

Editores Asociados

• Ricardo Berlanga • Emilio Lluis Puebla• Isaıas Lopez • Guillermo Pastor

• Vıctor Perez Abreu • Carlos Prieto• Carlos Renterıa • Luis Verde

Secretarias Tecnicas

• Roxana Martınez • Laura Valencia

Morfismos puede ser consultada electronicamente en “Revista Morfismos”de la direccion http://www.math.cinvestav.mx. Para mayores informes dirigirseal telefono 50 61 38 71.Toda correspondencia debe ir dirigida a la Sra. Laura Valencia, Departamentode Matematicas del Cinvestav, Apartado Postal 14-740, Mexico, D.F. 07000 opor correo electronico: [email protected].

VOLUMEN 7NÚMERO 1

ENERO A JUNIO DE 2003ISSN: 1870-6525

Informacion para Autores

El Consejo Editorial de MORFISMOS, Comunicaciones Estudiantiles del Departamentode Matematicas del CINVESTAV, convoca a estudiantes de licenciatura y posgrado a someterartıculos para ser publicados dentro de esta revista bajo los siguientes lineamientos

• Todos los artıculos seran enviados a especialistas para su arbitraje. No obstante, losartıculos seran considerados solo como versiones preliminares y por tanto pueden serpublicados en otras revistas especializadas.

• Se debe anexar junto con el nombre del autor, su nivel academico y la instituciondonde estudia o labora.

• El artıculo debe empezar con un resumen en el cual se indique de manera breve yconcisa el resultado principal que se comunicara.

• Es recomendable que los artıculos presentados esten escritos en Latex y sean enviadosa traves de un medio electronico. Los autores interesados pueden obtener el for-mato LATEX utilizado por MORFISMOS en “Revista Morfismos” de la direccion webhttp://www.math.cinvestav.mx, o directamente en el Departamento de Matematicasdel CINVESTAV. La utilizacion de dicho formato ayudara en la pronta publicaciondel artıculo.

• Si el artıculo contiene ilustraciones o figuras, estas deberan ser presentadas de formaque se ajusten a la calidad de reproduccion de MORFISMOS.

• Los autores recibiran un total de 15 sobretiros por cada artıculo publicado.

• Los artıculos deben ser dirigidos a la Sra. Laura Valencia, Departamento de Matemati-cas del Cinvestav, Apartado Postal 14 - 740, Mexico, D.F. 07000, o a la direccion decorreo electronico [email protected]

Author Information

MORFISMOS, the student journal of the Mathematics Department of Cinvestav, invitesundergraduate and graduate students to submit manuscripts to be published under thefollowing guidelines

• All manuscripts will be refereed by specialists. However, accepted papers will beconsidered to be “preliminary versions” in that authors may republish their papers inother journals, in the same or similar form.

• In addition to his/her affiliation, the author must state his/her academic status (stu-dent, professor,...).

• Each manuscript should begin with an abstract summarizing the main results.

• Morfismos encourages electronically submitted manuscripts prepared in Latex. Au-thors may retrieve the LATEX macros used for MORFISMOS through the web sitehttp://www.math.cinvestav.mx, at “Revista Morfismos”, or by direct request to theMathematics Department of Cinvestav. The use of these macros will help in theproduction process and also to minimize publishing costs.

• All illustrations must be of professional quality.

• 15 offprints of each article will be provided free of charge.

• Manuscripts submitted for publication should be sent to Mrs. Laura Valencia, De-partamento de Matematicas del Cinvestav, Apartado Postal 14 - 740, Mexico, D.F.07000, or to the e-mail address: [email protected]

Lineamientos Editoriales

“Morfismos” es la revista semestral de los estudiantes del Departamento deMatematicas del CINVESTAV, que tiene entre sus principales objetivos el que losestudiantes adquieran experiencia en la escritura de resultados matematicos.

La publicacion de trabajos no estara restringida a estudiantes del CINVESTAV;deseamos fomentar tambien la participacion de estudiantes en Mexico y en el extran-jero, ası como la contribucion por invitacion de investigadores.

Los reportes de investigacion matematica o resumenes de tesis de licenciatura,maestrıa o doctorado pueden ser publicados en MORFISMOS. Los artıculos queapareceran seran originales, ya sea en los resultados o en los metodos. Para juzgaresto, el Consejo Editorial designara revisores de reconocido prestigio y con experienciaen la comunicacion clara de ideas y conceptos matematicos.

Aunque MORFISMOS es una revista con arbitraje, los trabajos seconsideraran como versiones preliminares que luego podran aparecer pu-blicados en otras revistas especializadas.

Si tienes alguna sugerencia sobre la revista hazlo saber a los editores y con gustoestudiaremos la posibilidad de implementarla. Esperamos que esta publicacion pro-picie, como una primera experiencia, el desarrollo de un estilo correcto de escribirmatematicas.

Morfismos

Editorial Guidelines

“Morfismos” is the journal of the students of the Mathematics Department ofCINVESTAV. One of its main objectives is for students to acquire experience inwriting mathematics. MORFISMOS appears twice a year.

Publication of papers is not restricted to students of CINVESTAV; we want toencourage students in Mexico and abroad to submit papers. Mathematics researchreports or summaries of bachelor, master and Ph.D. theses will be considered forpublication, as well as invited contributed papers by researchers. Papers submittedshould be original, either in the results or in the methods. The Editors will assignas referees well–established mathematicians.

Even though MORFISMOS is a refereed journal, the papers will beconsidered as preliminary versions which could later appear in othermathematical journals.

If you have any suggestions about the journal, let the Editors know and we willgladly study the possibility of implementing them. We expect this journal to foster, asa preliminary experience, the development of a correct style of writing mathematics.

Morfismos

Contenido

The Lagrange approach to constrained Markov control processes: a survey andextension of results

Raquiel R. Lopez-Martınez and Onesimo Hernandez-Lerma . . . . . . . . . . . . . . . 1

Representaciones discretas en tiempo-frecuencia y el problema de la seleccionde frecuencias

Alin Andrei Carsteanu Manitiu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Generalized tilings with height functions

Olivier Bodini and Matthieu Latapy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

On Anosov energy levels that are of contact type

Osvaldo Osuna-Castro . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

Morfismos, Vol. 7, No. 1, 2003, pp. 1–26

The Lagrange approach to constrained Markovcontrol processes: a survey and extension of

results ∗

Raquiel R. Lopez–Martınez 1 Onesimo Hernandez–Lerma 2

Abstract

This paper considers constrained Markov control processes in Borelspaces, with unbounded costs. The criterion to be minimized is theexpected total discounted cost and the constraints are imposed onsimilar criteria. Conditions are given for the constrained problemto be equivalent to a convex program. We present a saddle-pointtheorem for the Lagrange function associated with the convex pro-gram, which is used to obtain the existence of an optimal solutionto the constrained problem. In addition, we show that there existsan optimal policy for the constrained problem which is also Paretooptimal for a certain multiobjective Markov control processes.

2000 Mathematics Subject Classification: 90C40, 93E20, 90C25.Keywords and phrases: Constrained Markov control processes, convexproblems, saddle point, Pareto policies.

1 Introduction

This paper gives a unified, self–contained presentation of constrainedMarkov control processes (MCPs) in Borel spaces with unbounded costs.The criterion to be minimized is an expected discounted cost and theconstraints are imposed on similar discounted cost functionals. The pa-per has two main objectives. First, it is a survey of several techniques to

∗Invited Article.1Research partially supported by a PROMEP grant.2Research partially supported by CONACyT Grant 37355–E.

1

2 R. R. Lopez–Martınez and O. Hernandez–Lerma

analyze constrained MCPs, with emphasis on the Lagrange approach.Second, it extends to constrained MCPs in general (i.e. nondenumer-able, noncompact) Borel spaces some results on the existence of optimalpolicies and it also studies the relation between the Lagrange and thePareto approaches. In particular, we show the existence of an optimalpolicy which is also Pareto optimal for a certain multiobjective MCP.

The constrained problem (CP) we are concerned with is of the fol-lowing form: given performance criteria V0, V1, . . . , Vq and constantsk1, . . . , kq,

Minimize V0(π)

over the set of control policies π that satisfy the constraints

Vi(π) ≤ ki ∀ i = 1, . . . , q.

Control problems of this form appear in many areas — see, for in-stance, [1–6, 8, 10–14, 19–25, 28–34]. The easiest way to analyze CP isusing the so–called direct method. In this method, which of course isalso applicable to unconstrained MCPs (e.g. [15], §5.7), the idea is touse occupation measures to transform CP into a “static” optimizationproblem, say CP’; see [13, 14] and §3 below. If one identifies the set ofoccupation measures with a convex subset of a suitable linear space of(signed) measures, then one can express CP’ in an obvious manner aseither a linear program or a convex program. The linear programmingformulation has been done for constrained MCPs in finite [8, 21, 22] orcountable [1–3,20] or even Borel [13,14] spaces. On the other hand, theconvex programming approach, which is the one we are interested in thispaper, was originally introduced by Beutler and Ross [4] for MCPs witha countable state space and a single constraint, but it has been extendedin many directions, for instance, countable state spaces with compactaction sets [1, 3, 5, 6, 31, 32] and Borel state spaces [23, 25, 27, 28, 33].(For the dynamic programming approach, which is not discussed in thispaper, see [29].)

As already mentioned above, in this paper we are mainly concernedwith the convex programming formulation of constrained MCPs withgeneral Borel state space and unbounded costs.

We begin in §2 by introducing some basic terminology and notation.In §3 we define the associated discounted occupation measures and stateLemma 3.3, which ensures that we can consider CP as a convex pro-gramming problem. In §4 we study the convex problem. In particular,we obtain a saddle-point theorem for the associated Lagrange function,

Constrained Markov control processes 3

which gives an optimal solution for CP. (A similar result for average costproblems appears in [25].) In §5 we establish some connections betweenthe Lagrange approach and the Pareto optimality of a certain multiob-jective MCP. Conditions are given under which an optimal policy forCP is Pareto optimal for the multiobjective problem. To illustrate theresults in §4 and §5, in §6 we study the so–called stochastic stabilizationproblem, from [9] and [27]. In particular, we show a saddle point forthe Lagrangean associated with this problem. In §7 and §8 we give theproof of Theorems 4.4, 4.5, and 5.4, which require lengthy preliminaries.

2 Constrained MCPs

Constrained MCPs are rather standard and will be introduced onlybriefly. (If necessary, see for instance [1, 13, 14, 28, 31, 32] for furtherdetails.)

The constrained Markov control model is of the form

(2.1) (X,A, A(x) |x ∈ X, Q, c,d,k),

where X and A are the state space and the control space, respectively.We shall assume that X and A are Borel spaces, endowed with thecorresponding Borel σ-algebras B(X), B(A). For each x ∈ X, thenonempty set A(x) in B(A) consists of the feasible controls or actionswhen the system is in state x ∈ X. We suppose that the set

(2.2) IK := (x, a) |x ∈ X, a ∈ A(x)

of feasible state-action pairs is a Borel subset of X × A. Moreover, Qstands for the transition law, and c : IK → IR is a measurable functionthat denotes the cost-per-stage . Finally, d = (d1, . . . , dq) : IK → IRq isa given function and k = (k1, . . . , kq) is a given vector in IRq, which areused to define the constrained problem (CP) in (2.5) and (2.6), below.

Let Π be the set of all (randomized, history-dependent) admisiblecontrol policies. Let Φ be the set of all the stochastic kernels ϕ on Agiven X such that ϕ(A(x)| x) = 1 for all x ∈ X, and let IF be the familyof measurable functions f : X → A for which f(x) ∈ A(x) for all x ∈ X. As usual, we will identify Φ with the family of randomized stationarypolicies, and IF with the subfamily of deterministic stationary policies.

Throughout the following, we consider a fixed discount factor δ ∈(0,1), and a fixed initial distribution γ0 ∈ IP(X), where IP(X) denotesthe set of probability measures on X. Given the functions c and d =


(d1, . . . , dq) as in (2.1), for each policy π ∈ Π, consider the expectedδ-discounted cost functions

(2.3) V0(π, γ0) := (1− δ)Eπγ0

! ∞"

t=0

δtc(xt, at)

#,

(2.4) Vi(π, γ0) := (1− δ)Eπγ0

! ∞"

t=0

δtdi(xt, at)

#for i = 1, . . . , q.

Furthermore, letting k = (k1, . . . , kq) be the q-vector in (2.1), define asubset ∆ of Π as

(2.5) ∆ := π | V0(π, γ0) < ∞ and Vi(π, γ0) ≤ ki (i = 1, . . . , q).

With this notation, we may then define the constrained problem (CP)we are concerned with as follows:

CP : Minimize V0(π, γ0)(2.6)

subject to π ∈ ∆.

If there exists a policy π∗ in ∆ that solves CP, that is,

(2.7) V0(π∗, γ0) = infV0(π, γ0) |π ∈ ∆ =: V ∗(γ0),

then π∗ is said to be an optimal policy for CP, and V ∗(γ0) is called theoptimal value of CP.

3 CP as a “static” optimization problem

The following conditions are used, in particular, to express CP asan optimization problem on a certain set of occupation measures —seeLemma 3.3.

Assumption 3.1

(a) The set IK (defined in (2.2)) is closed.

(b) c(x, a) is nonnegative and inf-compact, which means that for eachr ∈ IR the set (x, a) ∈ IK | c(x, a) ≤ r is compact.


(c) di(x, a) is nonnegative and lower semicontinuous (l.s.c.) for i =1, . . . , q.

(d) The transition law Q is weakly continuous, that is (denoting byCb(S) the space of continuous bounded functions on a topologicalspaces S), Q is such that

!X u(y)Q(dy|· ) belongs to Cb(IK) for

each function u in Cb(X).

(e) CP is consistent, that is, the set ∆ in (2.5) is nonempty.

Observe that Assumption 3.1(b) yields, in particular, that c is l.s.c.

Assumptions 3.1(b) and (c) can be replaced with the following: Thecost functions c and d1, . . . , dq are nonnegative and l.s.c., and at leastone of them is inf-compact. On the other hand, the “nonnegativity”condition on c and di may be replaced with “boundedness from below”.

Occupation measures. For each policy π ∈ Π, we define the oc-cupation measure µπ = µπ

γ0 as

(3.1) µπ(Γ) := (1− δ)∞"

t=0

δtP πγ0 [(xt, at) ∈ Γ] ∀Γ ∈ B(X ×A).

Then µπ is a probability measure (p.m.) onX×A, which is concentratedon IK, that is, µπ(IKc) = 0, where IKc stands for the complement of IK.Moreover, using the notation

⟨µ, h⟩ :=#

hdµ,

we can write (2.3) and (2.4) as

(3.2) V0(π, γ0) = ⟨µπ, c⟩ and Vi(π, γ0) = ⟨µπ, di⟩ (i = 1, . . . , q),

respectively.We shall denote by IP(IK) the set of p.m.’s on X × A that are con-

centrated on IK, and by IPOδ(IK) the subset of occupation measures.Further, for a p.m. µ in IP(IK), we denote by $µ its marginal on X, thatis, $µ(B) := µ(B ×A) for all B in B(X).

Remark 3.2 (See Remark 6.3.1 and Theorem 6.3.7 in [15].) For eachpolicy π ∈ Π, the occupation measure µπ ∈ IPOδ(IK) satisfies the follow-ing:

(3.3) %µπ(B) = (1− δ)γ0(B) + δ

#Q(B|x, a)µπ(d(x, a)) ∀B ∈ B(X).


Conversely, if µ is a p.m. in IP(IK) that satisfies (3.3), i.e.,

(3.4) !µ(B) = (1− δ)γ0(B) + δ

"Q(B|x, a)µ(d(x, a)) ∀B ∈ B(X),

then µ is in IPOδ(IK). In other words, there is a policy π for which µ isthe associated occupation measure, that is, µ = µπ. Therefore,

IPOδ(IK) = µ ∈ IP(IK) | µ satisfies (3.4).

We define the following subsets of IPOδ(IK):(3.5)

IPδ(IK) := µ ∈ IPOδ(IK)|⟨µ, c⟩ < ∞, and ⟨µ, di⟩ < ∞, i = 1, . . . q,

and

(3.6) ∆δ := µ ∈ IPδ(IK)| ⟨µ, di⟩ ≤ ki, i = 1, . . . q.

With this notation we can then state the following key fact.

Lemma 3.3 CP is equivalent to the problem:

CP′ : Minimize ⟨µ, c⟩subject to : µ ∈ ∆δ.

Proof: The lemma is a consequence of (3.2) and Remark 3.2. !

4 CP as a convex program

In Lemma 3.3 we already transformed CP into the “static” opti-mization problem CP′. We next use CP′ to restate CP as a convexprogram.

Let f and G be the functions on IPδ(IK) defined as

f(µ) := ⟨µ, c⟩ and G(µ) := (G1(µ), . . . , Gq(µ)),

with Gi(µ) := ⟨µ, di⟩−ki for i = 1, . . . , q. Obviously, f and G are convexfunctions. It is just as obvious that IPδ(IK) is a convex set. Thus, byLemma 3.3 we can represent CP as the convex problem

Minimize f(µ)(4.1)

subject to : µ ∈ IPδ(IK) and G(µ) ≤ θ,


where θ is the vector zero in IRq, and G(µ) ≤ θ means that Gi(µ) ≤ 0for all i = 1, . . . , q. Observe that the constraint in (4.1) can also bewritten as µ ∈ ∆δ.

The Lagrangean L : IPδ(IK)×IRq+ → IR associated with problem (4.1)

is given by

(4.2) L(µ,α) := f(µ) +G(µ) · α,

where α = (α1, . . . ,αq) is in IRq+, and “·” denotes the inner product in

IRq.

Remark 4.1 (a) ( See, for instance, [9, p. 88,89] or [18, p. 89]). If µ isin IP(IK), then there exists ϕ ∈ Φ such that µ can be “disintegrated” as

(4.3) µ(B × C) =

!

Bϕ(C|x)"µ(dx) ∀ B ∈ B(X), C ∈ B(A),

where "µ is the marginal of µ on X. In abbreviated form we write (4.3)as µ = "µ·ϕ.(b) If µ = "µ·ϕ is in IPOδ(IK), then it follows from (3.4) that µ is theoccupation measure of the policy ϕ ∈ Φ, that is, µ = µϕ.

The following saddle-point result gives conditions for problem (4.1)to have a solution.

Theorem 4.2 Suppose that there exists (µ∗,α∗) ∈ IPδ(IK) × IRq+ such

that the Lagrangean L has a saddle point at (µ∗,α∗), i.e.,

(4.4) L(µ∗,α) ≤ L(µ∗,α∗) ≤ L(µ,α∗)

for all (µ,α) in IPδ(IK)× IRq+. Then

(a) µ∗ solves problem (4.1), and(b) the disintegration µ∗ = #µ∗ · ϕ∗ of µ∗ satisfies that ϕ∗ is an optimalpolicy for CP.

Proof: The proof of part (a) is similar to that of Theorem 2 in [26, p.221], and, therefore, is omitted. Part (b) follows from (a), the Remark4.1(b), and the equivalence of CP and problem (4.1). !

In view of Theorem 4.2, to prove that the problem (4.1) is solvableit suffices to show the existence of a saddle point for L. This is true, inparticular, if the following condition holds.


Assumption 4.3 (Slater condition) There exists µ1 ∈ IPδ(IK) suchthat G(µ1) < θ, that is, Gi(µ1) < 0 for i = 1, . . . , q.

Theorem 4.4 Under Assumptions 3.1 and 4.3 , there exists a saddlepoint (µ∗,α∗) for the Lagrangean L, and, therefore, CP is solvable.

Proof: See §7. !

To summarize, Theorem 4.4 gives the existence of a saddle point(µ∗,α∗) for L, which, by Theorem 4.2 yields an optimal policy ϕ∗ forCP. It turns out that the converse is also true, as shown in the followingresult.

Theorem 4.5 Suppose that Assumptions 3.1 and 4.3 hold. If µ∗ =!µ∗ · ϕ∗ ∈ ∆δ is such that ϕ∗ is an optimal policy for CP, then theLagrangean L has a saddle point.

Proof: See §7. !

Remark 4.6 (See Remark 4.2.5, p. 51 in [7].) In our present con-text, Assumption 4.3 is equivalent to the so-called Karlin condition (orconstraint qualification), according to which there is no nonzero vectorα ∈ IRq

+ for which G(µ) · α ≥ 0 for all µ ∈ IPδ(IK).

5 The Lagrange approach vs Pareto optimality

In this section we compare the Lagrange approach to CP with thePareto optimality of a certain multiobjective MCP. With this in mind,we first briefly introduce multiobjective MCPs (for more informationsee, for instance [17] or [28]).

Let V0(π, γ0) and Vi(π, γ0) be as in (2.3) and (2.4), and let V (π, γ0) ∈IRq+1 be the cost vector

(5.1) V (π, γ0) := (V0(π, γ0), . . . , Vq(π, γ0)).

The multiobjective control problem we are concerned with is to finda policy π∗ that “minimizes” V (·, γ0) in the sense of Pareto. To statethis in precise form, we first simplify the notation by writing V (π, γ0)simply as V (π).


Definition 5.1 Let Γ(Π) ⊂ IRq+1 be the set of cost vectors in (5.1),i.e.,

Γ(Π) := V (π) | π ∈ Π,

which is sometimes called the performance set of the multiobjectiveMCP. Then a policy π∗ is said to be Pareto optimal (or a Pareto policy)if there is no π ∈ Π such that V (π) = V (π∗) and Vi(π) ≤ Vi(π∗) forall i = 0, . . . , q. The set of cost vectors in Γ(Π) corresponding to Paretopolicies is called the Pareto set of Γ(Π), and it is denoted by Par(Γ(Π)).

Let IRq+1++ be set of vectors in IRq+1 with strictly positive components.

Let β ∈ IRq+1++ , and consider the scalar (or real-valued) cost-per -stage

function

(5.2) Cβ(x, a) := β0c(x, a) +q!

i=1

βidi(x, a),

and the δ-discounted cost V β(π) = V β(π, γ0) with

(5.3) V β(π) := (1− δ)Eπγ0

" ∞!

t=0

δtCβ(xt, at)

#.

Using (5.1) and (5.2) we may write V β(π) as

(5.4) V β(π) = β · V (π) =q!

i=0

βiVi(π).

Let

(5.5) Λ := β ∈ IRq+1++ |

q!

i=0

βi = 1.

We may then obtain the existence of Pareto policies by the standard“scalarization” approach, as follows.

Theorem 5.2 Choose an arbitrary vector β ∈ Λ. If π∗ ∈ Π is anoptimal policy for the scalar criterion (5.3), that is,

(5.6) V β(π∗) ≤ V β(π) ∀ π ∈ Π,

then π∗ is Pareto optimal.


For a proof of Theorem 5.2 see, for instance, Theorem 3.2(a) in [17].In general, the constrained problem CP in (2.6) can have optimal

policies that are not Pareto optimal. On the other hand, if CP hasa unique optimal policy π∗, then it is easily seen (directly from theDefinition 5.1) that π∗ is a Pareto policy. The following two theoremsgive other cases in which an optimal policy for CP is in fact a Paretopolicy.

Theorem 5.3 Let (µ∗,α∗) ∈ IPδ(IK) × IRq++ be a saddle point for the

Lagrangean L, and disintegrate µ∗ as !µ∗ ·ϕ∗. Then ϕ∗ is Pareto optimal.

Proof: From the definition (4.4) of a saddle point, we have that

(5.7) L(µ∗,α∗) ≤ L(µ,α∗) ∀ µ ∈ IPδ(IK).

On the other hand, from (3.2) and the definition (4.2) of L it followsthat

(5.8) L(µ,α) = V0(π) +q"

i=1

αi(Vi(π)− ki),

where π is a policy associated to the occupation measure µ. Hence,from (5.7) and (5.8) we have that

V0(ϕ∗) +

q"

i=1

α∗i (Vi(ϕ

∗)− ki) ≤ V0(π) +q"

i=1

α∗i (Vi(π)− ki) ∀ π ∈ Π.

Equivalently, defining β∗ := (1,α∗) ∈ IRq+1++ , we have

β∗ · V (ϕ∗)− α∗ · k ≤ β∗ · V (π)− α∗ · k ∀ π ∈ Π,

and so

(5.9) β∗ · V (ϕ∗) ≤ β∗ · V (π) ∀ π ∈ Π.

Finally, let P = 1 +#q

i=1 α∗i . Then, multiplying both sides of (5.9) by

1/P , it follows from Theorem 5.2 that ϕ∗ is Pareto optimal. !

Now consider the following subset of Γ(Π)

(5.10) Γ∗(Π) := V (π) | π an optimal policy for CP.

Let Par(Γ∗(Π)) be the Pareto set of Γ∗(Π).


Theorem 5.4 Under Assumption 3.1, the Pareto set Par (Γ∗(Π)) ofΓ∗(Π) is nonempty.

Proof: See §8. !It turns out that the nonemptiness of Par (Γ∗(Π)) in Theorem 5.4

ensures the existence of a Pareto policy that is optimal for CP.

Theorem 5.5 Under Assumptions 3.1 and 4.3, there exists an optimalpolicy π∗ for CP, which is also Pareto optimal.

Proof: From Theorem 5.4 there exists a policy π∗ such that V (π∗) isin Par(Γ∗(Π)). By (5.10), π∗ is an optimal policy for CP. We nowclaim that π∗ is Pareto optimal, that is, V (π∗) is in Par(Γ(Π)). Indeed,if π∗ is not Pareto optimal, then there exists a policy π1 ∈ Π suchthat V (π1) = V (π∗) and Vi(π1) ≤ Vi(π∗) for i = 0, . . . , q. Hence,V (π1) ∈ Γ∗(Π), which contradicts our assumption on π∗. Therefore, π∗

is Pareto optimal. !

6 Example

To illustrate the results in Sections 4 and 5, we next consider the fol-lowing problem, which is similar to the stochastic stabilization problemin [9, 27]. First, we show that Assumptions 3.1 and 4.3 hold. Then, weprove that this problem is solvable using the Lagrange approach, thatis, we shall obtain a saddle point for the Lagrange function. Finally, weconstruct the corresponding Pareto set. For notational ease, we shallwrite the δ-discounted costs in (2.3) and (2.4) without the factor (1−δ).

Consider the scalar linear system

(6.1) xt+1 = xt − at + ξt for t = 0, 1, . . . ,

with state and control spaces X = A = IR. The disturbances ξt are i.i.d.random variables, independent of the initial state x0, and such that

(6.2) E(ξ0) = 0 and E(ξ20) =: σ2 < ∞.

Let c(x, a) and d(x, a) be the quadratic costs defined as

(6.3) c(x, a) = x2 + a2, d(x, a) = (x− a)2,


and consider the following constrained problem in which k is a givenpositive constant.

Minimize V0(π, γ0) := Eπγ0

! ∞"

t=0

δt(x2t + a2t )

#

subject to : V1(π, γ0) := Eπγ0

! ∞"

t=0

δt(xt − at)2

#≤ k.

It is clear that the Assumptions 3.1(a), (b), (c) are satisfied in this ex-ample. Moreover, by the continuity of the right-hand side of (6.1) withrespect to xt and at for every ξt it follows that also Assumption 3.1(d)holds. On the other hand, if we take π = f0 ∈ IF as the “identity”policyf0(x) := x for all x ∈ X, we see that V1(f0, x) = 0, and, therefore, As-sumptions 3.1(e) and 4.3 are both satisfied. Summarizing, Assumptions3.1 and 4.3 hold for this problem.

Now, from (3.2) and (4.2) the corresponding Lagrange function is

(6.4) L(π,α) = V0(π, γ0) + (V1(π, γ0)− k) · α

with α ≥ 0. Let

(6.5) L1(α) := infπ∈Π

L(π,α).

Note that defining the new cost per-stage function

Cα(x, a) := c(x, a) + α · d(x, a) = x2 + a2 + α(x− a)2

and denoting by V α(π, γ0) the corresponding δ-discounted cost, we mayexpress (6.4) as

L(π,α) = V α(π, γ0)− α · k.

Therefore, finding a policy that attains the minimun in (6.5) becomes alinear-quadratic problem; see, for instance, p. 162 in [9], p. 70 in [15],or p. 253 in [28]. From any of these references we have

infπ∈Π

V α(π, x)− k · α = z(α)v(x)− k · α ∀ x ∈ X,

with v(x) := x2+(1− δ)−1δσ2, and z(α) is the maximal solution of thequadratic equation

(6.6) δz2 + (1 + α− 2δ)z − 1− 2α = 0.


Therefore, assuming that the initial distribution γ0 satisfies that

(6.7) γ0 :=

!v(x)γ0(dx) < ∞,

we can express (6.5) as

(6.8) L1(α) = z(α)γ0 − k · α.

Moreover, the deterministic stationary policy fα ∈ IF given by

(6.9) fα(x) =α+ δz(α)

1 + α+ δz(α)x

is optimal for V α(π, x) for all x ∈ IR, and so we also have L1(α) =L(fα,α) for each α ≥ 0. Now, to obtain a saddle point for the La-grangean in (6.4) we first prove the following, which can be seen as an“explicit” form of Lemma 7.2, below.

Proposition 6.1 If the constraint constant k satisfies the inequality

(6.10) 0 < k < K,

where K := γ0(1 + 2δ −√1 + 4δ2)/2δ

√1 + 4δ2, then there exists a unique

α∗ > 0 such thatL1(α

∗) = maxα≥0

L1(α).

Proof: We differentiate the function L1 in (6.8) with respect to α, toget L′

1(α) = z′(α)γ0 − k.Let us now show that L′

1(α) = 0 has a unique positive solution.With this in mind, first note that the positive solution of (6.6) is

z(α) =−(1 + α− 2δ) +

"(1 + α− 2δ)2 + 4δ(1 + 2α)

2δ.

Hence

z′(α) = − 1

2δ+

1 + α+ 2δ

2δ"(1 + α− 2δ)2 + 4δ(1 + 2α)

,

and so

(6.11) L′1(α) =

#− 1

2δ+

1 + α+ 2δ

2δ"(1 + α− 2δ)2 + 4δ(1 + 2α)

$γ0 − k.


According to (6.10) and (6.11) we have

L′1(0) =

γ0(1 + 2δ −√1 + 4δ2)

2δ√1 + 4δ2

− k > 0.

On the other hand,limα→∞

L′1(α) = −k < 0.

Hence the equation L′1(α) = 0 has a positive solution. Moreover, from

(6.11), L′1(α) = 0 becomes

(1 + α+ 2δ)2 = 4δ2(k(γ0)−1 + (2δ)−1)2((1 + α− 2δ)2 + 4δ(1 + 2α)).

As this equation is quadratic in α, it has a unique positive solution. !Let α∗ be as in Proposition 6.1 and define z∗ = z(α∗) and f∗ := fα∗

as in (6.9), that is,

f∗(x) := fα∗(x) = (α∗ + δz∗)(1 + α∗ + δz∗)−1x.

Then (f∗,α∗) is a saddle point for L, and, therefore, from Theorem 4.2it follows f∗ is an optimal policy for CP. Moreover, as α∗ is positive,from Theorem 5.3 we have that f∗ is Pareto optimal.

Remark 6.2 If α = 0, then f∗0 (x) = δz0x(1 + δz0)−1 is optimal for V0,

that is,V0(f

∗0 , γ0) = inf

π∈ΠV0(π, γ0)

where z0 is the positive solution of the quadratic equation

(6.12) δz2 + (1− 2δ)z − 1 = 0.

On the other hand, we can see that the “identity” policy f0(x) = x isoptimal for V1, and obviously, V1(f0, γ0) = 0, that is,

infπ∈Π

V1(π, γ0) = 0.

Proposition 6.3 Let !f be a constant, and f ∈ IF a stationary policygiven by f(x) := !fx for all x ∈ X. Let θ := 1− !f . If |θ| < 1, then

(6.13) V0(f, γ0) =1 + !f

2

1− δθ2γ0,

(6.14) V1(f, γ0) =(1− !f )2

1− δθ2γ0.


In particular, for K and f∗0 as in (6.10) and Remark 6.2,

(6.15) V1(f∗0 , γ0) = K.

Proof: Replacing at in (6.1) with at = f(xt) = !fxt, we obtain

xt = (1− !f)xt−1 + ξt−1 = θxt−1 + ξt−1 ∀ t = 1, 2, . . . .

Hence, for all t = 1, 2, . . .

xt = θtx0 +t−1"

j=0

θjξt−1−j ,

and so

Efx (x

2t ) = θ2tx2 +

σ2(1− θ2t)

1− θ2.

This yields that

(6.16) Efx

# ∞"

t=0

δtx2t

$=

1

1− δθ2

%x2 +

σ2δ

1− δ

&=

v(x)

1− δθ2.

Hence, from (6.7),

(6.17) Efγ0

# ∞"

t=0

δtx2t

$=

γ01− δθ2

.

Now note that using a = f(x) = !fx in (6.3) we get

(6.18) c(x, a) = (1 + !f2)x2 and d(x, a) = (1− !f)2x2

for all x. Thus, inserting (6.17) and (6.18) in V0 and V1 we obtain (6.13)and (6.14). Finally, from (6.14) and Remark 6.2 we have

(6.19) V1(f∗0 , γ0) =

γ0(1 + δz0)2 − δ

.

On the other hand, from (6.12) we get

(6.20) z0 =2δ − 1 +

√1 + 4δ2

2δ.

Hence, substituting (6.20) in (6.19) we obtain (6.15). !


Remark 6.4 Suppose that instead of (6.10) we have

k ≥ K,

and let f∗0 (x) = δz0x(1+δz0)−1 be the optimal policy for V0 (see Remark

6.2). Then, from (6.15) it follows that

V1(f∗0 , γ0) = K ≤ k

and, therefore, f∗0 is an optimal policy for the constrained problem.

Moreover, f∗0 is the unique optimal policy for CP, and so it is Pareto

optimal, that is, (V0(f∗0 , γ0), V1(f∗

0 , γ0)) belongs to the Pareto set. (SeeFigure 6.1.)

The Pareto set. We next construct the Pareto set in an explicit form.As seen above, f∗ is an optimal policy for CP which is also Paretooptimal, that is, (V0(f∗, γ0), V1(f∗, γ0)) is in the Pareto set. Whenthe constraint constant k varies in the interval (0,K), with K as in(6.10), then (V0(f∗, γ0), V1(f∗, γ0)) describes the Pareto set. Obviously,V0(f∗, γ0) is the optimal value for the constrained problem, that is,V0(f∗, γ0) = V ∗(γ0). Now, we wish to find the value of V1(f∗, γ0).

Proposition 6.5 For each k as in (6.10),

V1(f∗, γ0) := Ef∗

γ0

! ∞"

t=0

δt(xt − at)2

#= k

and so (V0(f∗, γ0), V1(f∗, γ0)) = (V ∗(γ0), k) belongs to the Pareto set.

Proof: Since (f∗,α∗) is a saddle point and f∗ is an optimal policy forCP we have

V ∗(γ0) ≤ L(f∗,α∗) = V ∗(γ0) + (V1(f∗, γ0)− k)α∗.

On the other hand, as (V1(f∗, γ0)− k)α∗ ≤ 0, it follows that

V ∗(γ0) + (V1(f∗, γ0)− k)α∗ ≤ V ∗(γ0)

and so we have (V1(f∗, γ0) − k)α∗ = 0. This equality together withProposition 6.1 yields that V1(f∗, γ0) = k. !

Proposition 6.5 ensures that (V ∗(γ0), k) belongs to the Pareto setwhen k varies in (0,K). Furthermore, if α∗ is as in Proposition 6.1,


it is clear then that V ∗(γ0) = L1(α∗). Now, in connection with theFigure 6.1, let us fix w = k and calculate y = L1(α∗). First, we notethe following facts. Proposition 6.3 yields that

(6.21) V1(f∗, γ0) =

γ0(1 + α∗ + δz∗)2 − δ

.

Further, from (6.6) with α = α∗ we have

(6.22) α∗ =δ(z∗)2 + (1− 2δ)z∗ − 1

2− z∗

and subtituting this value of α∗ in (6.21) it follows that

(6.23) V1(f∗, γ0) =

(2− z∗)2γ01− δ(2− z∗)2

.

Hence, from Proposition 6.5 we get

(6.24)(2− z∗)2γ0

1− δ(2− z∗)2= k.

Now, substituting 6.22 in L(α∗) we obtain

y = z∗γ0 −δ(z∗)2 + (1− 2δ)z∗ − 1

2− z∗k

=−(z∗)2(γ0 + δk) + 2z∗(γ0 + δk)− kz∗ + k

2− z∗.(6.25)

From (6.24) we have that

(6.26) −(z∗)2(γ0 + δk) = 4(γ0 + δk)(1− z∗)− k.

The latter equality together with (6.25) gives

(6.27) y = 2(γ0 + δk)− z∗

2− z∗k.

Now let

s :=z∗

2− z∗.

Solving this equation for z∗ and substituting the solution in (6.24) weget

4γ0(1 + s)2 − 4δ

= k,


which yields

(6.28) w = k = γ04

(s+ 1)2 − 4δ

and so, from (6.25),

(6.29) y = 2(γ0 + kδ)− 4δγ0(s+ 1)2 − 4δ

.

In (6.28) and (6.29) s is the parameter which varies as k is in (0,K).The graph of (6.28)-(4.29) is the Pareto set, which is represented inFigure 6.1.

V0 (f0 , γ0 ) V( γ0 )*

( f0 ,γ0 )V1

y

w

0Par Γ( (Π))

*

k

Γ ( Π)

γ2 0

Figure 6.1


7 Proof of Theorems 4.4 and 4.5

The proof of Theorems 4.4 and 4.5 is based on the following prelim-inary facts. Consider the functions

(7.1) L1(α) := infµ∈IPδ(IK)

L(µ,α),

(7.2) L2(µ) := supα≥θ

L(µ,α),

and let V ∗(γ0) be as in (2.7). Note that, by Lemma 3.3,

V ∗(γ0) = inf⟨µ, c⟩ | µ ∈ ∆δ.

Remark 7.1 As ∆δ ⊂ IPδ(IK), for each α ∈ IRq+ we have

L1(α) ≤ infµ∈∆δ

L(µ,α) ≤ infµ∈∆δ

⟨µ, c⟩ = V ∗(γ0),

that is, L1(α) ≤ V ∗(γ0) for all α ∈ IRq+. Similarly, V ∗(γ0) ≤ L2(µ) for

all µ ∈ ∆δ. Hence

(7.3) supα≥θ

L1(α) ≤ V ∗(γ0) ≤ infµ∈IPδ(IK)

L2(µ).

The following lemmas show that equality holds throughout (7.3).

Lemma 7.2 Under Assumptions 3.1 and 4.3, there exists α∗ in IRq+

such thatL1(α

∗) = supα≥θ

L1(α) = V ∗(γ0).

Proof: In the space IR× IRq define the sets

B1 := (r,α)| r ≥ f(µ),α ≥ G(µ) for some µ ∈ IPδ(IK) ,B2 := (r,α)| r ≤ V ∗(γ0),α ≤ θ .

The set B2 is obviously convex, and so is B1 because f and G are convex.By definition of V ∗(γ0), the set B1 contains no interior points of B2.On the other hand, it is clear that B2 contains an interior point. Thus,by the Separating Hyperplane Theorem (see, for example, [26], p. 133,Theorem 3), there is a vector (r∗,α∗) ∈ IR× IRq such that

r∗r1 + α1 · α∗ ≥ r∗r2 + α2 · α∗


for all (r1,α1) ∈ B1 and all (r2,α2) ∈ B2. By the definition of B2 itfollows that r∗ ≥ 0,α∗ ≥ θ. We next show that in fact r∗ > 0. Indeed,as the vector (V ∗(γ0), θ) is in B2, we have

(7.4) r∗r + α · α∗ ≥ r∗V ∗(γ0)

for all (r,α) ∈ B1. Thus, if r∗ = 0, then α · α∗ ≥ 0 for all α ∈ IRq

such that (r,α) ∈ B1. In particular, taking α = G(µ1) with µ1 as inAssumption 4.3, we obtain G(µ1)·α∗ ≥ 0, which implies that Gi(µ1) ≥ 0for some i = 1, . . . , q. As this contradicts Assumption 4.3, it follows thatr∗ > 0 and, without loss of generality, we may assume r∗ = 1.

Now, since the point (V ∗(γ0), θ) is in the closure of both B1 and B2,we have (with r∗ = 1 in (7.4))

V ∗(γ0) = inf(r,α)∈B1

[r + α · α∗] ≤ infµ∈IPδ(IK)

[f(µ) +G(µ) · α∗]

= infµ∈IPδ(IK)

L(µ,α∗) ≤ infµ∈∆δ

f(u) = V ∗(γ0).

Hence, recalling (7.3) the lemma is proved. !

By (7.1) and (7.2) the following lemma is a “minimax” result.

Lemma 7.3 Under Assumptions 3.1 and 4.3, we have

(7.5) maxα≥θ

L1(α) = infµ∈IPδ(IK)

L2(µ) = V ∗(γ0).

Proof: Since G(µ) · α ≤ θ for all µ ∈ ∆δ and α ≥ 0, we see that

L2(µ) = supα≥θ

L(µ,α) = ⟨µ, c⟩ for all µ ∈ ∆δ.

Henceinf

µ∈∆δ

L2(µ) = V ∗(γ0).

It follows thatinf

µ∈IPδ(IK)L2(µ) ≤ V ∗(γ0),

and so, by ( 7.3) and Lemma 7.2, the equality (7.5) holds. !

Lemma 7.4 Under Assumption 3.1, there exists a p.m. µ∗ in IPδ(IK)such that

L2(µ∗) = inf

µ∈IPδ(IK)L2(µ) = V ∗(γ0).


Proof: If µ is in IPδ(IK) but not in ∆δ, then there exists i0 in 1, . . . , qsuch that Gi0(µ) > 0, which implies that L2(µ) = +∞. Therefore,

(7.6) infµ∈IPδ(IK)

L2(µ) = infµ∈∆δ

L2(µ) = V ∗(γ0).

On the other hand, for all µ ∈ ∆δ and α ≥ θ, we have G(µ) · α ≤ 0,and so it follows that

(7.7) L2(µ) = supα≥θ

L(µ,α) = ⟨µ, c⟩ ∀ µ ∈ ∆δ.

Therefore, from the (7.6), (7.7), together with Lemma 3.3 and Theorem3.2 in [13], the desired conclusion follows. !

We are now ready for the proof of Theorems 4.4 and 4.5.Proof of Theorem 4.4. Let α∗ and µ∗ be as in Lemma 7.2 and Lemma7.4, respectively. From lemma 7.3 we have that

L(µ∗,α∗) = V ∗(γ0).

Now, by the latter equality together with Lemmas 7.2, 7.4 and thedefinition of L1 and L2 it follows that

L(µ∗,α∗) = L1(α∗) ≤ L(µ,α∗) for all µ ∈ IPδ(IK),

and, similarly,

L(µ∗,α∗) = L2(µ∗) ≥ L(µ∗,α) for all α ≥ θ.

Therefore, the pair (µ∗,α∗) is a saddle point. !

Proof of Theorem 4.5. Let α∗ be as in Lemma 7.2. As G(µ∗) ≤ 0and f(µ∗) = V ∗(γ0), it follows that

L(µ∗,α∗) ≤ L(µ,α∗) for all µ ∈ IPδ(IK),

which gives the second inequality in (4.4). On other hand, since

V ∗(γ0) ≤ f(µ∗) +G(µ∗) · α∗ ≤ f(µ∗) = V ∗(γ0),

we have G(µ∗) · α∗ = 0. Therefore,

L(µ∗,α)− L(µ∗,α∗) = G(µ∗) · α−G(µ∗) · α∗ = G(µ∗) · α ≤ 0,

and the first inequality in (4.4) follows. !


8 Proof of Theorem 5.4

For completeness, we first state some well-known results that areneeded to prove Theorem 5.4.

Lemma 8.1 Let Y be a metric space and M a family of probabilitymeasures on Y. If there exists a nonnegative and inf-compact functionv on Y such that

sup⟨µ, v⟩, µ ∈ M < ∞,

then M is relatively compact, that is, for each sequence µn in M thereis a probability measure µ on Y and a subsequence µm of µn suchthat µm converges weakly to µ in the sense that

(8.1) ⟨µm, v⟩ → ⟨µ, v⟩ ∀v ∈ Cb(Y ).

To prove Lemma 8.1, one first shows that the hypothesis impliesthat M is tight, and then the relative compactness of M follows fromProhorov’s Theorem (see [16]).

Lemma 8.2 Let Y a metric space, and v : Y → IR lower semicontin-uous and bounded below. If µn and µ are probability measures on Yand µn converges weakly to µ (that is, as in (8.1)), then

lim infn→∞

⟨µn, v⟩ ≥ ⟨µ, v⟩.

Lemma 8.2 is well known (and easy to prove): see, for instance,statement (12.3.37) in [16, p. 243]

Lemma 8.3 The set IPδ(IK) is closed with respect to the topology ofweak convergence.

For a proof of Lemma 8.3 see Lemma 5.5 in [17], for instance.

Let

∆′δ := µ ∈ ∆δ | µ is an optimal solution for (4.1).

Lemma 8.4 Let V β(π) and Γ∗(Π) be as in (5.3) and (5.10), respec-tively. Let Π∗ be set of policies π such that V (π) in Γ∗(Π). Then thereexists a policy π∗ such that

(8.2) V β(π∗) = minπ∈Π∗

V β(π).


Proof: It is clear that minimizing V β(·) on Γ∗(Π) is equivalent to min-

imizing ⟨·, Cβ⟩ on ∆′δ, with Cβ as in (5.2). Let ρ∗ := inf⟨µ,Cβ⟩ | µ ∈

∆′δ and take a sequence µn in ∆′

δ such that

⟨µn, Cβ⟩ ↓ ρ∗.

Therefore, given ϵ > 0, there exists an integer N such that

(8.3) ρ∗ ≤ ⟨µn, Cβ⟩ ≤ ρ∗ + ϵ ∀n ≥ N.

On the other hand, by definition of ∆′δ, it follows that

(8.4) ⟨µn, c⟩ = V ∗(γ0) for all n ≥ 0

with V ∗(γ0) as in (2.7), which implies that

supn⟨µn, c⟩ = V ∗(γ0).

Since c is inf-compact (Assumption 3.1(b)), from Lemma 8.1 it followsthat µn is relatively compact, that is, there exists a probability mea-sure µ∗ on IK and a subsequence µm of µn that converges weakly toµ∗. The latter convergence, together with (8.3) and Lemma 8.2, yields

that ⟨µ∗, Cβ⟩ = ρ∗. Finally, from Lemma 8.3 we conclude that µ∗ isindeed a p.m. in ∆′

δ, and so the disintegration µ∗ = !µ∗ ·ϕ∗ of µ∗ is suchthat π∗ := ϕ∗ satisfies (8.2). !

Proof of Theorem 5.4 From Lemma 8.4 and Theorem 5.2, it followsthat Par(Γ∗(Π)) = ∅. !

Raquiel R. Lopez–MartınezFacultad de MatematicasUniversidad VeracruzanaA. P. 270Xalapa, Ver., [email protected]

Onesimo Hernandez–LermaDepartamento de MatematicasCINVESTAV–IPNA. P. 14–740Mexico D.F., [email protected]

References

[1] Altman E., Constrained Markov Decision Processes, Chapman &Hall /CRC, Boca Raton, FL, 1999.


[2] Altman E., Constrained Markov decision processes with total costcriteria: occupation measures and primal LP, Math. Meth. Oper.Res., 43 (1996), 45-72.

[3] Altman E., Constrained Markov decision processes with total costcriteria: Lagrange approach and dual LP,Math. Meth. Oper. Res.,48 (1998), 387-417.

[4] Beutler F. J.; Ross K. W., Optimal policies for controlled Markovchains with a constraint, J. Math. Anal. Appl., 112 (1983), 236-252.

[5] Borkar V.S., A convex analytic approach to Markov decision pro-cesses, Prob. Theory Related Fields, 78 (1988), 583-602.

[6] Borkar V.S., Ergodic control of Markov chains with constraints—the general case, SIAM J. Control Optim., 32 (1994), 176-186.

[7] Craven B. D., Mathematical Programming and Control Theory,Chapman and Hall, London, 1978.

[8] Derman B. D.; Veinott A. F. Jr., Constrained Markov decisionchains, Management Science, 19 (1972), 389-390.

[9] Dynkin E.B.; Yushkevich A. A., Controlled Markov Processes,Springer-Verlag, Berlin, 1979.

[10] Feinberg E.; Shwartz A., Constrained discounted dynamic pro-gramming, Math. Oper. Res., 21 (1996), 922-945.

[11] Feinberg E.; Shwartz A., Constrained dynamic programming withtwo discount factors: applications and an algorithm, IEEE Trans.Autom. Control, 44 (1999), 628-631.

[12] Golabi K.; Kulkarni R.B.; Way G.B., A statewide pavement man-agement system, Interfaces, 12 (1982), 5-21.

[13] Hernandez–Lerma O.; Gonzalez-Hernandez J., ConstrainedMarkov control processes in Borel spaces: the discounted case,Math. Meth. Oper. Res., 52 (2000), 271-285.

[14] Hernandez–Lerma O.; Gonzalez-Hernandez J.; Lopez-MartınezR.R., Constrained average cost Markov control processes in Borelspaces, SIAM J. Control Optim., (to appear).


[15] Hernandez–Lerma O.; Lasserre J.B., Discrete-Time Markov Con-trol Processes: Basic Optimality Criteria, Springer-Verlag, NewYork, 1996.

[16] Hernandez–Lerma O.; Lasserre J.B., Further Topics on Discrete-Time Markov Control Processes, Springer-Verlag, New York,1999.

[17] Hernandez–Lerma O.; Romera R. , Multiobjective Markov controlprocesses: a linear programing approach, preprint, Departamentode Matematicas, CINVESTAV-IPN, Mexico, 2003. (Submitted).

[18] Hinderer K., Foundations of Non-stationary Dynamic Program-ming with Discrete-Time Parameter, Lecture Notes Oper. Res.Math. Syst. 33, Springer-Verlag, Berlin, 1970.

[19] Hordijk A.; Spieksma F., Constrained admission control to aqueueing system, Adv. Appl. Prob., 21 (1989), 409-431.

[20] Huang Y.; Kurano M., The LP approach in average rewards MDPswith multiple cost constraints: The countable state case, J. Inform.Optim. Sci., 18 (1997), 33-47.

[21] Kallenberg L. C. M., Linear Programming and Finite MarkovianControl Problems, Mathematical Centre Tracts 148, Amsterdam,1983.

[22] Kallenberg L. C. M., Survey of linear programming for standardand nonstandard Markovian control problems, Part I: Theory,ZOR–Math. Methods Oper. Res., 40 (1994), 1-42.

[23] Kurano M., Nakagami J.; Y. Huang, Constrained Markov decisionprocesses with compact state and action spaces: the average case,Optimization, 48 (2000), 255-269.

[24] A. Lazar, Optimal flow control of a class of queueing networks inequilibrium, IEEE Trans. Autom. Control, 28 (1983), 1001-1007.

[25] Lopez–Martınez R.R., A saddle–point theorem for constrainedMarkov control processes, Morfismos, 3 (1999), 69–79.(Available at http://chucha.math.cinvestav.mx/morfismos/v3n2/index.html)


[26] Luenberger D. G., Optimization by Vector Space Methods, Wiley,New York, 1969.

[27] Mao X.; Piunovskiy A.B., Strategic measure in optimal controlproblems for stochastic sequences, Stoch. Anal. Appl., 18 (2000),755-776.

[28] Piunovskiy A.B., Optimal Control of Random Sequences in Prob-lems with Constraints, Kluwer, Boston, 1997.

[29] Piunovskiy A.B.; Mao X., Constrained Markovian decision pro-cesses: the dynamic programming approach, Oper. Res. Letters,27 (2000), 119-126.

[30] Ross K.; Varadarajan R., Multichain Markov decision processeswith a sample path constraint: a decomposition approach, Math.Oper. Res., 16 (1991), 195-207.

[31] Sennott L.I., Constrained discounted Markov decision chains,Prob. Eng. Inform. Sci., 5 (1991), 463-475.

[32] Sennott L.I., Constrained average cost Markov decision chains,Prob. Eng. Inform. Sci., 7 (1993), 69-83.

[33] Tanaka K., On discounted dynamic programming with constraints,J. Math. Anal. Appl., 155 (1991), 264-277.

[34] Vakil F.; Lazar A. A., Flow control protocols for integrated net-works with partially observed voice traffic, IEEE Trans. Autom.Control, 32 (1987), 2-14.


Representaciones discretas en tiempo–frecuenciay el problema de la seleccion de frecuencias ∗

uitinaMunaetsraCierdnAnilA

Resumen

El artıculo presenta la problematica general de la obtencion debases en Rn con significado de frecuencia. Tambien se presenta elefecto sobre la caracterıstica de contraste de una base, de la mini-mizacion de una funcion objetivo aditiva (tal como lo serıa unafuncion de costo), definida sobre los elementos de aquella base. Sediscuten varios algoritmos de optimizacion, basados en el paquetede ondıculas. Finalmente, se propone una solucion novedosa alo que es el problema del calculo de un espectro energetico signi-

aadinetboesabanunelanesanuednoicatneserperaledovitacfipartir del paquete de ondıculas.

2000 Mathematics Subject Classification: 42C40, 65T60.Keywords and phrases: ondıcula, base ortonormal, representacion tiem-po–frecuencia, analisis armonico, minimizacion discreta, contraste.

1 Introduccion

,selanesednoicatneserperalaacidedesaicneucerf–opmeitnesisilanalEcon soporte contiguo o discreto, en terminos de un diccionario de formas

y”ateludno“noslonapsenesodasusonimretsortO.salucıdnouadnoed“ondeleta”, imitaciones onomatopeyicas del frances “ondelette”. Prefe-rimos “ondıcula”, con el cual se guarda el sentido original de la pal-

ocisalcoditneslenesenoicnufedsedresnedeup”selanes“saL.arbahasta distribuciones y procesos estocasticos. Tambien llamado analisis

∗Artıculo invitado.

27

28 Alin Andrei Carsteanu Manitiu

armonico (por razones historicas, dado que las primeras funciones anal-izadoras fueron funciones armonicas, siendo este el caso de las trans-formadas de Fourier), el analisis en tiempo–frecuencia tiene sus raıcesen la relacion complementaria que guardan el tiempo y la frecuencia.Dicha relacion tambien llama la atencion a la localizacion en tiempo–frecuencia de los elementos de la representacion, aspecto que se discutiraa continuacion, ası como a la posibilidad de obtener algoritmos para eluso de funciones analizadoras de formas arbitrarias (en vista que en latransformada de Fourier, el algoritmo hace uso de las propiedades dela exponencial compleja, de las cuales resulta la simetrıa de la trans-formada inversa con respecto a la directa). Ası, dado el significado deamplitudes correspondientes a diferentes frecuencias, que adquieren loscoeficientes de las representaciones en tiempo–frecuencia, varias formasde las funciones analizadoras tienen un interes particular, de acuerdoa su interpretacion fısica, habiendose compilado hasta diccionarios defunciones analizadoras. Tambien, el analisis en tiempo–frecuencia se ex-tiende de manera natural a soportes multidimensionales, que pueden serasociados, en aplicaciones, con espacios fısicos bidimensionales o tridi-mensionales, espacio–tiempos u otras entidades. El presente artıculo selimitara, de acuerdo a su tıtulo, al soporte unidimensional discreto.

Consideremos una funcion f definida sobre un soporte discreto, for-mado por n puntos equidistantes: f = [f (τ) , f (2τ) , f (3τ) , . . . , f (nτ)],donde n es una potencia de 2. En las aplicaciones, tal f resulta general-mente de un muestreo. El proposito declarado del analisis armonico esde encontrar una descomposicion de f de la forma

f =n!

k=1

ckwk,(1)

donde laswk = [wk (τ) , wk (2τ) , . . . , wk (nτ)] son n ondıculas (funcionesanalizadoras) de una familia y los ck son los coeficientes (o amplitudes)correspondientes a cada ondıcula. Si las ondıculas se escogen de talmanera que los coeficientes ck sean unicamente determinados, entoncesla familia de ondıculas constituye una base en Rn (f es un vector real n-dimensional). Las ondıculas se llaman “atomos” en tiempo frecuencia,en el sentido de que unas son versiones re–escalonadas y/o trasladadasde las otras, ocupando por convencion distintos rectangulos, de areasiguales, en el plano tiempo–frecuencia:

wk (t) =1

√ak

W

"t− bkak

#,(2)

Representaciones discretas en tiempo–frecuencia 29

donde W es la ondıcula–base, los ak son factores de escala y los bk sonfactores de posicion en su argumento.

En el presente artıculo se usaran ondıculas de Haar-Walsh [3, 9],por sus cualidades de localizacion, presentadas en la siguiente seccion.A continuacion se presentara el algoritmo de paquetes de ondıculas, quepermite construir varias bases de ondıculas a partir de una ondıcula deorigen. Finalmente, se mostraran los algoritmos que permiten escogeruna base optima con respecto a una funcion objetivo definida sobre elconjunto de los coeficientes, ası como el contenido en frecuencias dedichas bases y algunas de sus aplicaciones.

2 Paquetes de ondıculas

Fieles al proposito de descomponer la senal f en funcion de las frecuen-cias presentes en ella, consideremos dos operadores, un operador A de“alta frecuencia” y un operador B de “baja frecuencia”, definidos sobreR2 de la siguiente manera:

!A [x, y] = (x− y)/

√2

B [x, y] = (x+ y)/√2.

(3)

Observemos que al aplicar los operadores A y B a una senal f re-sulta [A,B] f = [Af , Bf ] = [A [f1, f2] , . . . , A [fn−1, fn] , B [f1, f2] , . . . ,B [fn−1, fn]] y se obtiene una base en Rn (recordemos que n es divisiblepor 2, siendo una potencia de 2), dado que los operadores son linealese invertibles, de manera que

!(A [x, y] +B [x, y])/

√2 = x

(A [x, y]−B [x, y])/√2 = y.

(4)

Es facil verificar que esta base es ortonormal. Observemos tambien queen el cambio de base, a cada par de coordenadas en la base temporal lecorresponde un par de coordenadas en la nueva base. Por lo tanto, lascoordenadas de cada par en la nueva base cubren exactamente el doblede la extension temporal de una coordenada en la base temporal (elperıodo del muestreo), y difieren entre sı unicamente en su contenido defrecuencia. Una representacion intuitiva de la posicion de los elementosde las dos bases en el plano tiempo–frecuencia, para n = 8, se muestraen la figura 1.

El sistema de funciones ortogonales (3) fue propuesto por primeravez por Haar [3] y Walsh [9].


f1 f2 f3 f4 f5 f6 f7 f8

t

ω

Base temporal

Bf12 Bf34 Bf56 Bf78

Af12 Af34 Af56 Af78

t

ω

Nueva base

Figura 1: Localizacion en el plano tiempo–frecuencia de las coordenadasen la base temporal y en la nueva base obtenida por la aplicacion de losoperadores [A,B].

Aplicando de nuevo los operadores A y B a cada uno de Af y Bf ,obtenemos la siguiente base, representada en la figura 2. A cada unode BAf , AAf , ABf y BBf se le pueden aplicar una ultima vez losoperadores A y B, ya que en cada uno de los vectores antes mencionadosquedan dos elementos. El resultado es la base frecuencial (o base deFourier), representada en la misma figura 2.

BBf1−4 BBf5−8

ABf1−4 ABf5−8

AAf1−4 AAf5−8

BAf1−4 BAf5−8

t

ω

Siguiente base

BBBf1−8

ABBf1−8

AABf1−8

BABf1−8

BAAf1−8

AAAf1−8

ABAf1−8

BBAf1−8

t

ω

Base frecuencial

Figura 2: Localizacion en el plano tiempo–frecuencia de las coordenadasen la siguiente base obtenida por la aplicacion de los operadores [A,B],ası como en la base frecuencial.

Las 4 bases representadas formalmente (ası como graficamente) paran = 8 en las figuras 1 y 2 tienen en comun el hecho de que dentro decada una de ellas, los operadores A o B aparecen un mismo numero


de veces en cada elemento. Por lo tanto, llamaremos a las bases conesta propiedad “bases fundamentales”. En el caso mas general en quen es una potencia cualquiera de 2, hay log2 (n) + 1 bases fundamen-tales, correspondientes al aplicar los operadores 0, 1, . . . , log2 (n) veces.Notemos que cada base fundamental es exhaustiva con respecto a las2k permutaciones posibles (con repeticiones) de los dos operadores Ay/o B, aplicados k veces, sobre un subconjunto de n/2k puntos deldominio de definicion de f . Por lo tanto, el conjunto de bases funda-mentales contiene todos los elementos de bases que se puedan obteneraplicando los operadores A y/o B; llamamos a este conjunto el “paquetede ondıculas”.

Una observacion importante para la obtencion de diferentes bases us-ando los operadores A y B es el hecho siguiente: debido a que la trans-formacion (3) involucra cada vez solamente 2 elementos de una base,sobre cada subconjunto m2k +1, . . . , (m+ 1) 2k, k ∈ 1, . . . , log2 (n)y m ∈ 0, . . . , n/2k−1, del dominio de definicion de f se pueden aplicarlos operadores A o B, en cualquier orden, cualquier numero de vecesentre 0 y k, y el resultado serıa de nuevo una base en Rn (al ser lineale invertible la transformacion (3), la ortonormalidad de tales bases esmenos trivial y sera discutida aparte). Un ejemplo de base construida deeste modo es la base–ondıcula (representada en la figura 3 para n = 8).La propiedad distintiva de esta base es que, siendo sus operadores (dealtas a bajas frecuencias) A, AB, ABB, ABBB etc., en cada banda defrecuencia, la escala (el factor ak de la ecuacion (2), o equivalentementeel numero de elementos de la base temporal involucrados en la repre-sentacion del elemento en cuestion de la presente base) es inversamenteproporcional con la frecuencia de aquella banda. Historicamente, estafue la primera base en tiempo–frecuencia desarrollada a parte de lasbases fundamentales.

BBBf1−8

ABBf1−8

ABf1−4 ABf5−8

Af12 Af34 Af56 Af78

t

ω

Figura 3: Localizacion en el plano tiempo–frecuencia de las coordenadasen la base–ondıcula.


La base frecuencial resulta, si se toma la base temporal como baseestandar, despues de O (n log (n)) operaciones. Este orden de velocidades similar a la del algoritmo de la transformada rapida de Fourier [2],dado que siguiendo la definicion se trata de n convoluciones de la funcionf , definida en n puntos, con diferentes ondıculas de este mismo tamano,tales como son representadas en la figura 4.

Algoritmo discreto de Fourier Algoritmo del paquete de ondıculasω0

ω1

ω2

ω3

ω4

Figura 4: Formas de las ondas analizadoras correspondientes a dife-rentes frecuencias ω para el algoritmo del paquete de ondıculas (derecha)comparadas con las del algoritmo discreto de Fourier (izquierda), ambosalgoritmos usan la funcion analizador de Haar.

Wickerhauser [10] demuestra la equivalencia, en primer orden (y porlo tanto, tambien asıntoticamente, para n grande), entre el algoritmo delos paquetes de ondıculas y el algoritmo de Fourier. Ademas, se puede


construir un paquete similar al de ondıculas aplicando el algoritmo deFourier sucesivamente a los n, n/2 + n/2, n/4 + n/4 + n/4 + n/4, etc.puntos del dominio de definicion de f . Sin embargo, el algoritmo deFourier es “rapido” solamente para bases trigonometricas, mientras queel algoritmo presentado en esta seccion mantiene el mismo O (n log (n))para cualquier par de operadores [A,B].

Demostraremos a continuacion la ortonormalidad de las bases cons-truidas a partir del paquete de ondıculas Haar. (a) Por su modo deconstruccion, que consiste en aplicar el conjunto de operadores [A,B]a subconjuntos m2k + 1, . . . , (m + 1)2k, para k ∈ 1, . . . , log2 (n) ym ∈ 0, . . . , n/2k − 1 del dominio de definicion de f , dos elementosarbitrarios de una base obtenida a partir del paquete de ondıculas Haartienen que ser ajenos o en sus representaciones por elementos de la basetemporal, o en sus representaciones por elementos de la base frecuencial(o en ambos casos). Supongamos, sin perdida de generalidad, que seaeste el caso de la base temporal, y consideremos la misma base tempo-ral como base estandar. (Podemos hacer la suposicion sin perdida degeneralidad debido a que el producto de A y B aplicados a un par de vec-tores unitarios [1k,1j ] resulta ser A [1k,1j ] ·B [1k,1j ] =

121

2k−

121

2j = 0.)

Entonces los dos elementos son trivialmente ortogonales. (b) La norma-lidad de las bases esta garantizada por el hecho que los operadores [A,B]conservan las normas en L2: (A [fk, fj ])

2+(B [fk, fj ])2 = 1

2 (fk + fj)2+

12 (fk − fj)

2 = f2k + f2

j .

Notemos que la ortonormalidad de los operadores A, B no es sufi-ciente para la ortonormalidad de toda base ası obtenida del paquete deondıculas; sino que es necesario que los elementos involucrados en cadaaplicacion de los operadores sean los mismos como en el caso de losoperadores Haar-Walsh. De lo contrario, al aplicar la transformacionde base, por intermedio de tales operadores, a un numero de n ele-mentos, su norma llega a encontrarse, despues de la transformacion, enun numero de elementos mayor que n, o bien al restringirnos a n ele-mentos, la norma de estos disminuirıa generalmente con respecto a lanorma inicial. Estos efectos son llamados efectos terminales, y si no sonmuy importantes en magnitud, se dice que las respectivas bases son casiortonormales. Varios autores (por ejemplo, Herley et al. [5]) proponenpara estos casos la ortonormalizacion de las bases, por procedimientostipo Gramm-Schmidt o similares. Notemos que con este proceder sepierde el significado en tiempo–frecuencia de dichas bases. (Mientrasque este significado no es importante, por ejemplo, para fines de com-


presion de senales, para la mayorıa de las aplicaciones de investigacionsı lo es.)

3 Optimalidad de las bases construidas del pa-quete de ondıculas

3.1 Criterios globales y funciones objetivo locales

El proposito de esta optimizacion es de escoger la base que minimiza unafuncion objetivo, definida sobre los coeficientes de la representacion de fen la base, de tal manera que se optimicen ciertas caracterısticas cualita-tivas globales de dicha representacion. Tal caracterıstica es, por ejemplo,el llamado “contraste”. Intuitivamente, aumentar el contraste quiere de-cir aumentar los elementos mayores y disminuir los elementos menores.En terminos de una base que resultarıa en la representacion mas con-trastante de un vector dado, se elige de un conjunto de bases aquella quemaximiza las coordenadas mayores (en valor absoluto) del vector con-siderado (la ortonormalidad de las bases asegura la minimizacion de lascoordenadas menores). En las aplicaciones, maximizar el contraste esutil para fines de compresion de senales, haciendo posible la eliminacionde un mayor numero de elementos pequenos (compresion aproximada)o ceros (compresion exacta), ası como para fines de investigacion en laestructura en tiempo–frecuencia de las senales, revelando las frecuenciasdominantes por un perıodo limitado de tiempo.

Una manera para definir el contraste en una representacion es la deordenar las coordenadas de manera decreciente y construir la secuenciade normas (usuales, en L2) de los vectores formados por las primeras1, 2, . . . , n coordenadas, lo que es equivalente (para bases ortonormales)con la secuencia de sumas parciales de los cuadrados de las primeras1, 2, . . . , n coordenadas. Por su construccion, esta secuencia representauna funcion no convexa, llamemosla σ (k) en funcion del numero k decoordenadas involucradas, haciendo la convencion σ (0) = 0. Podemosdefinir la superioridad del contraste de una primera representacion encomparacion con una segunda representacion: si σ1 (k) ≥ σ2 (k), ∀k ∈1, . . . , n, y ∃k ∈ 1, . . . , n tal que σ1 (k) > σ2 (k). Observemos que,en el caso general, esta definicion no introduce un orden completo en elconjunto de representaciones: las graficas de las funciones σ (k) de dosdeterminadas representaciones se pueden cruzar, en cuyo caso ningunade las dos (de acuerdo a esta definicion) tiene un contraste superior a la


otra. Un modo sencillo de resolver este problema serıa de fijar un ciertok0, para que sea el unico punto en donde comparamos σ1 (k0) y σ2 (k0).Para algunas aplicaciones en particular, fijar un tal k0 podrıa ser util,pero aquı preferimos analizar el caso general, aun con la desventajade no tener un orden completo de las representaciones por medio delcontraste.

Queda entonces encontrar una funcion objetivo aditiva, definida so-bre los coeficientes de la representacion de f en una base, de tal maneraque se optimice el contraste. Demostraremos en este contexto el si-guiente teorema:

Teorema 3.1.1 Sean las representaciones de una senal f en dos basesortonormales distintas, con una de las representaciones mas contrastadaque la otra, en el sentido que σ1 (k) ≥ σ2 (k), ∀k ∈ 1, . . . , n, y ∃k ∈1, . . . , n tal que σ1 (k) > σ2 (k), y sea s (·) una funcion objetivo real,concava. Entonces

n!

k=1

s"#c2k$1

%<

n!

k=1

s"#c2k$2

%.

Demostracion: Probamos que se puede construir una operacion quetransforma la secuencia ordenada |ck|1 a |ck|2 en un numero finitode pasos, tal que en cada paso ocurre una disminucion de

&nk=1 s

"c2k%.

Seaδ := min

k∈l,...,m−1[σ1 (k)− σ2 (k)]

tal que δ > 0, σ1 (l − 1) = σ2 (l − 1) y σ1 (m) = σ2 (m), n ≥ m > l > 0.La existencia de una tal subsecuencia l, . . . ,m − 1 esta garantizadapor σ1 (0) = σ2 (0) = 0, σ1 (n) = σ2 (n) por ortonormalidad de lasbases, y ∃k ∈ 1, . . . , n tal que σ1 (k) > σ2 (k). Cambiemos entonces#c2l$2←

#c2l$2+δ y

#c2m

$2←

#c2m

$2−δ, lo que significa σ2 (k)← σ2 (k)+δ,

∀k ∈ l, . . . ,m− 1. Como previsto, encontramos una disminucion de&nk=1 s

"#c2k$2

%: s

"#c2l$2+ δ

%+ s

"#c2m

$2− δ

%< s

"#c2l$2

%+ s

"#c2m

$2

%,

como δ > 0, l ≥ m ⇐⇒ [cl]2 ≥ [cm]2, y la funcion s es concava. Eneste momento, para por lo menos un punto k0 ∈ l, ...,m− 1 tenemosσ1 (k0) = σ2 (k0). Podemos repetir este procedimiento para subsecuen-cias terminando, y respectivamente empezando, en k0. La no-conve-xidad y la monotonicidad de σ seran respetadas dentro de las subse-cuencias, como solamente hay una traslacion vertical, y seran tambienrespetadas alrededor de k0, como σ2 (k0 − 1) ≤ σ1 (k0 − 1) ≤ σ1 (k0) =


σ2 (k0) ≤ σ2 (k0 + 1) ≤ σ1 (k0 + 1). Por lo tanto, la existencia y el or-den de [ck]2 sera preservado a cada paso, realizando una disminucion de!n

k=1 s"#c2k$2

%al respectivo paso siguiente, hasta que

maxk∈1,...,n

σ1 (k)− σ2 (k) = 0 ⇐⇒ σ1(k) = σ2(k), ∀k ∈ 1, . . . , n

⇐⇒ [ck]1 = [ck]2, ∀k ∈ 1, . . . , n,

es decir que las dos representaciones se vuelven identicas y el algoritmose termina (despues de un numero de pasos menor o igual a n). !

3.2 Algoritmos de optimizacion

En la seccion precedente hemos mostrado como escoger una funcionobjetivo, definida sobre las coordenadas de f en una base, y cuya mini-mizacion optimice el contraste de la representacion de f en dicha base.Queda entonces disenar algoritmos para buscar dentro del paquete deondıculas la base ortonormal que minimice una funcion objetivo dada.La busqueda esta basada en el hecho que para realizar cambios de baseusando los operadores A y B sobre 2 coordenadas temporalmente con-secutivas de una base, se puede reemplazar este par de coordenadas conun par de coordenadas vecinas en frecuencia, y recıprocamente, comose mostro en la seccion 2.

El primer algoritmo de minimizacion de la funcion objetivo, ac-tuando solamente sobre un subconjunto de bases del paquete de ondıcu-las, se encuentra en Wickerhauser [10], y se conoce como el algoritmo(sencillo) del arbol (binario).

Para ilustrar el algoritmo, es util poner las representaciones en basestiempo–frecuencia de las figuras 1 y 2 en forma del arbol de la figura5, en el cual cada nivel es la representacion de la senal en una base,correspondiendo (de abajo hacia arriba) de base temporal hasta la base

BBBf ABBf AABf BABf BAAf AAAf ABAf BBAf

BBf ABf AAf BAf

Bf Af

f

Figura 5: Arbol del paquete de ondıculas.


frecuencial. (Observemos que para n > 23, hay mas niveles en el arbol,hacia arriba, para llegar a la base frecuencial y completar todo el pa-quete de ondıculas.) Para escoger los elementos de la base optimizadaen este caso, empezando con la base frecuencial, se compara la funcionobjetivo para cada par de coordenadas adyacentes en frecuencia con elpar que les corresponde por la ecuacion (3) (es decir que se comparanlos elementos correspondientes entre la penultima y la ultima base – lafrecuencial), escogiendo cada vez entre los dos pares aquel que minimicela funcion objetivo. A continuacion, se comparan cuartetos de coorde-nadas de la misma frecuencia en la antepenultima base con los cuartetoscorrespondientes de la base obtenida al paso anterior, escogiendo entrelos cuartetos correspondientes cada vez aquel que minimice la funcionobjetivo. Despues, se comparan octetos de coordenadas de la mismafrecuencia en la precedente base con los octetos correspondientes de labase obtenida al paso anterior, y ası sucesivamente hasta la base tempo-ral. El algoritmo es facil de seguir, de arriba hacia abajo, sobre el arbolde la figura 5. Sin embargo, aunque el algoritmo minimice la funcionobjetivo sumada sobre los elementos de la base (con las consecuenciasglobales discutidas en la seccion precedente), se observa inmediatamenteque este algoritmo no tiene acceso a todas las bases que se puedan cons-truir a partir del paquete de ondıculas; ademas, induce una asimetrıaartificial entre tiempo y frecuencia.

El mismo algoritmo puede ser empleado empezando con la base tem-poral (llamemoslo algoritmo inverso del arbol sencillo). Aunque esto nomejora el algoritmo, ha sugerido la idea para otro algoritmo, operandosobre un conjunto sensiblemente mas grande de bases, y publicado porHerley et al. [4, 5], ası como por Ramchandran y Vetterli [7]. El llamadoalgoritmo doble del arbol consta en correr el algoritmo sencillo a partirde cada una de las bases fundamentales (correspondientes a cada unode los niveles del arbol), teniendo en consecuencia O

!n log2 (n)

"opera-

ciones. El algoritmo guarda las mismas desventajas que el algoritmosencillo, en el sentido que todavıa no tiene acceso a todas las basesque puedan resultar del paquete de ondıculas (lo que resultara obvioen la presentacion de los siguientes algoritmos), y tambien guarda unaasimetrıa entre tiempo y frecuencia (por correr hacia la base temporal,o posiblemente en el otro sentido, pero no en ambos).

Finalmente, algoritmos que escogen entre todas las bases posiblesdel paquete de ondıculas fueron construidos independientemente porThiele y Villemoes [8], Herley et al. [6] y Carsteanu et al. [1]. Estosalgoritmos estan basados en la misma idea (presentada a continuacion),


diferenciandose solamente en el manejo de las variables en memoria(para minimizar la memoria ocupada [1] o el tiempo de ejecucion delalgoritmo [8, 6], respectivamente). Observemos primero que las basesortonormales que se pueden construir por las relaciones (3) y (4) a partirdel paquete de ondıculas no pueden contener simultaneamente elementosde la base temporal y de la base frecuencial (por no ser ortogonalesentre sı). Esta constatacion ofrece la clave para construir un algoritmoque tenga acceso a todas las bases ortonormales que se puedan cons-truir a partir del paquete de ondıculas, y de ella resulta directamente elsiguiente lema:

Lema 3.2.1 Sea P (1, . . . , n) el conjunto de bases ortonormales enRn que se pueden construir a partir del respectivo paquete de ondıculas,F (1, . . . , n) el subconjunto de P (1, . . . , n) de bases que no con-tienen ningun elemento de la base temporal, y T (1, . . . , n) el subcon-junto de P (1, . . . , n) de bases que no contienen ningun elemento dela base frecuencial. Entonces

F (1, . . . , n) ∪ T (1, . . . , n) = P (1, . . . , n) .

De este lema desprendemos que cualquier base ortonormal constru-ida a partir del paquete de ondıculas (y por lo tanto la base optima bus-cada) pertenece a F (1, . . . , n) o a T (1, . . . , n) (o a los dos), por locual la busqueda algorıtmica de la base optima dentro de P (1, . . . , n)se reduce a dos busquedas dentro de F (1, . . . , n) y de T (1, . . . , n),respectivamente. Obviamente, para n > 2, tenemos F (1, . . . , n) ∩T (1, . . . , n) = ∅, lo que significa que se puede mejorar el algoritmo debusqueda en terminos de rapidez (a precio de usar mas memoria) memo-rizando F (1, . . . , n) ∩ T (1, . . . , n) y buscando una sola vez en estesubconjunto (en lugar de dos veces, al buscar dentro de F (1, . . . , n) ydentro de T (1, . . . , n)). Thiele y Villemoes [8] muestran que memo-rizando todas las comparaciones hechas en cada etapa, el algoritmo llegaa tener O (n log (n)) operaciones. Veremos a continuacion dos lemas,que nos permitiran continuar la busqueda de la base optima dentro deT (1, . . . , n) y dentro de F (1, . . . , n), respectivamente:

Lema 3.2.2 Las bases del subconjunto T (1, . . . , n) contienen sola-mente elementos de las primeras n − 1 bases del respectivo paquete.Por lo tanto, tenemos que P (1, . . . , n/2) × P (n/2 + 1, . . . , n) =T (1, . . . , n). Es decir que T (1, . . . , n) consta del paquete constru-ido de los primeros n/2 elementos de la base temporal y del paqueteconstruido de los ultimos n/2 elementos de dicha base.


Lema 3.2.3 Las bases del subconjunto F (1, . . . , n) contienen sola-mente elementos de las ultimas n−1 bases del respectivo paquete. Por lotanto, F (1, . . . , n) consta del paquete construido de los primeros n/2elementos de la base temporal y del paquete construido de los ultimosn/2 elementos de dicha base.

Hemos entonces reducido, en 2 pasos intermedios, la busqueda dela base optima dentro de P (1, . . . , n) a la busqueda dentro de 4 pa-quetes de bases de dimension n/2. Como el conjunto generado por los4 paquetes es identico a P (1, . . . , n) (cualquier base que exista enP (1, . . . , n) se encuentra tambien en el conjunto), concluimos que elalgoritmo toma en cuenta todas las bases ortonormales que se puedanconstruir del paquete de bases de dimension n y, por induccion, que elalgoritmo encuentra el resultado despues de O

!n2

"pasos (sin tomar

en cuenta ningun modo de eliminar las redundancias entre F (·) y T (·)mencionadas anteriormente).

Observemos sin embargo que # (P (1, . . . , n)), el numero de basescontenidas en P (1, . . . , n), no crece con n de manera polinomial,sino exponencial, lo que harıa un procedimiento por comparaciones di-rectas practicamente imposible de usar. Efectivamente, como por ra-zones de simetrıa tenemos # (P (1, . . . , n)) = 2#2 (P (1, . . . , n/2))−#4 (P (1, . . . , n/4)), y como # (P (1)) = 1 y # (P (1, 2)) = 2, ten-emos que ∃β tal que limn→∞ #(P (1, . . . , n))/βn =

!√5− 1

"#2, con

β ≈ 1.84454757 . . . [1]. Para tener una idea del numero de bases quepueden tomar en cuenta los algoritmos antes presentados, mencionemosque para el algoritmo del arbol doble, β ≈ 1.7148445 . . ., mientras quepara el algoritmo del arbol sencillo, β ≈ 1.5028368 . . . (el lımite es iguala 1 en los dos ultimos casos) [8]. Para precisar esta comparacion visuali-zando graficamente ciertas diferencias entre los algoritmos, en la figura6 se presentan (para n = 8) las localizaciones en tiempo–frecuencia de:(A) una base accesible a los tres algoritmos; (B) una base que no puedeser tomada en cuenta por el algoritmo del arbol sencillo; y (C) unabase a la cual no tienen acceso ni el algoritmo del arbol sencillo, ni elalgoritmo del arbol doble.

4 Aplicaciones

Como ya discutido, maximizar el contraste de una senal es util para finesde compresion, haciendo posible la eliminacion de un mayor numero de


t

ω

At

ω

B

t

ω

C

Figura 6: Representaciones en tiempo–frecuencia de: (A) una base ac-cesible a los tres algoritmos; (B) una base que no puede ser tomada encuenta por el algoritmo del arbol sencillo; y (C) una base a la cual notienen acceso ni el algoritmo del arbol sencillo, ni el algoritmo del arboldoble.

elementos pequenos para compresion aproximada, o ceros para com-presion exacta. En la figura 7, presentamos la base temporal, ası comola base optima de una senal de 256 muestras. La funcion objetivo es-cogida es la entropıa, la senal llegando de una entropıa de 5.1444 . . . enla base temporal a una entropıa de 1.1858 . . . en la base optima. Dadoque la entropıa es una funcion concava, se realiza tambien un contrastemas fuerte en la base optima (figura 8). Este contraste nos permite re-construir aproximadamente la senal usando solamente 20 coordenadasde la base optima (figura 9).

Para fines de investigacion en la estructura en tiempo–frecuenciade una senal, una representacion optima puede revelar las frecuenciasdominantes por un perıodo limitado de tiempo, lo que no es realizableni en la representacion temporal, ni en la representacion frecuencial. Lafigura 10 nos muestra la representacion en su base optima de un ejemplode una tal senal con frecuencia variable en el tiempo, un chirrido.


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 50 100 150 200 250 3000

50

100

150

200

250

300

t

f

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 50 100 150 200 250 3000

50

100

150

200

250

300

t

f

Figura 7: Representacion de una senal de 256 muestras en la base tem-poral (izquierda), ası como la base optima (derecha): las amplitudes(coordenadas en las respectivas bases) estan representadas por intensi-dad de gris.

5 El problema de las frecuencias para el espec-tro energetico

En las secciones precedentes hemos visto como, a partir del paquete deondıculas, se pueden construir bases en Rn, con significado en tiempo–frecuencia, que minimicen una funcion aditiva sobre los elementos de labase, y en consecuencia adquieren ciertas caracterısticas de contraste.Sin embargo, una vista mas cuidadosa al paquete de ondıculas reve-la que, igual que en el caso del algoritmo (real) discreto de Fourier,los pares de elementos consecutivos en frecuencia (con excepcion delprimero y del ultimo elemento) de cualquier base construida, son carac-terizados en realidad por la misma frecuencia, y difieren solamente poruna traslacion (bk de la ecuacion (2)). Se presenta entonces el problemade como obtener la norma de las coordenadas correspondientes a cadafrecuencia, el conjunto de dichas normas constituye el llamado “espec-tro energetico”. Dicho espectro, interpretado como “contenido” de en-ergıa en cada frecuencia (tambien “densidad” de energıa en el caso deespectros continuos) tiene varias aplicaciones en procesos fısicos carac-terizados por frecuencias. Para ejemplificar, el espectro energetico deuna senal que representa la variacion en el tiempo de la velocidad en unpunto de un fluido turbulento es gobernado por las leyes de escalamientoque rigen el mismo fluido, y son el resultado de la transferencia de ener-gıa dentro del rango inercial de escalas del fluido, desde las escalas de


100 101 102 1030

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Número de elementos de la base

Frac

ción

de

la n

orm

a L 2 re

cupe

rada

Figura 8: Fraccion de la norma L2 de la senal reconstruida, en funciondel numero de las coordenadas mayores usadas en la reconstruccion:notemos la superioridad del contraste de la base optima (lınea inte-rrumpida) comparado con el contraste de la base temporal (lınea con-tinua).

inyeccion de energıa hacıa el rango viscoso. El estudio del respectivoespectro energetico ha permitido explicar, aunque todavıa no completa-mente, el fenomeno de movimiento turbulento. Por lo tanto, estimamoscomo muy importante la posibilidad de extraer el espectro energeticode una senal a partir de las bases construidas en tiempo–frecuencia.

Para poder evaluar todas las normas correspondientes a frecuenciasen una base, en ella cada elemento de la base temporal debe entrar enel calculo de elementos de una sola representacion fundamental. El con-junto de bases que satisfacen esta condicion es precisamente el conjuntode bases accesibles por el algoritmo inverso del arbol sencillo (recorde-mos que por “inverso” hemos entendido el intercambio de tiempo confrecuencia en el algoritmo usual). Esto, porque en el algoritmo del arbolsencillo, la descomposicion frecuencial es constante en el tiempo [10], ypor lo tanto, en el algoritmo inverso, la descomposicion temporal es lamisma para todas las frecuencias. Lo que quiere tambien decir que para


0 50 100 150 200 250 3000

0.2

0.4

0.6

0.8

1

1.2

1.4

0 50 100 150 200 250 3000

0.2

0.4

0.6

0.8

1

1.2

1.4

Figura 9: La senal original de 256 muestras (izquierda) y la senal re-construida de 20 coordenadas de la base optima (derecha).

las bases obtenidas por los demas algoritmos, en general no se puedecalcular un espectro energetico con significado.

Proponemos aquı una vision diferente acerca de este asunto: calcularel espectro energetico con solamente n/2 + 2 en lugar de n elementos.El siguiente lema nos ayuda a hacerlo:

Lema 5.1.1 Usando el algoritmo para obtener la base optima, si para-mos la descomposicion en frecuencia a 4 elementos en lugar de 2, ysi anulamos para propositos de comparacion los 2 elementos extremosde los 4 (aparte del caso que son los mismos extremos de una basefundamental), procuramos comparar todas las bases construidas de estamanera, a partir del paquete de ondıculas.

Para probarlo, es suficiente observar que el algoritmo es equivalenteal aplicar el algoritmo usual al conjunto F descrito en la seccion anterior.

La ventaja de este acercamiento radica en el hecho que para lamayorıa de las senales, las n/2 + 2 coordenadas mayores de la baseoptima contienen casi toda la energıa de la senal, mientras que la baseoptima tiene en la mayorıa de los casos un σ (k) superior a lo de labase obtenida por el algoritmo inverso del arbol sencillo, para todok ∈ 1, . . . , n/2 + 2.

6 Conclusiones

Hemos presentado aquı una revision del estado actual de los algoritmosde optimizacion discreta para representaciones en tiempo–frecuencia,


0 5 10 15 20 25 30 35−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 5 10 15 20 25 30 350

5

10

15

20

25

30

35

t

f

Figura 10: Un chirrido y su representacion en la base optima (dondelas coordenadas estan representadas por intensidad de gris): observe-mos como las frecuencias dominantes son delimitadas durante diversosintervalos de tiempo.

enfocando el trabajo en ejemplificar las caracterısticas especiales y loslogros de cada algoritmo. Una atencion particular fue extendida al pro-blema de la seleccion de frecuencias, la cual juega un papel importanteen la definicion del espectro energetico de una senal. En este contexto,observamos que una version nueva de un algoritmo existente puede re-solver el problema, y tambien proponemos un algoritmo nuevo, basadoen un acercamiento diferente, que surge de los lemas comprobados enel presente artıculo. Si bien este algoritmo resulta superior en terminosde contraste, la cuestion de la optimizacion del contraste entre todas lasbases que permiten la definicion de un espectro energetico queda comoproblema abierto para el futuro.

AgradecimientosEl autor quiere agradecer a los editores de Morfismos la invitacion

a escribir el presente artıculo, ası como a los tres revisores anonimosquienes, con sus comentarios, contribuyeron a mejorar el texto, y aJesus Gonzalez Espino Barros por su revision final.

Alin Carsteanu ManitiuDepartamento de Matematicas,CINVESTAV-IPNA.P. 14-740Mexico D.F. [email protected]


Referencias

[1] Carsteanu A.; Sapozhnikov V. B.; Venugopal V.; Foufoula-Georgiou E., Absolute optimal time-frequency basis – a researchtool, J. Phys. A: Math. Gen., 30 (1997), 7133-7146.

[2] Cooley J. W.; Tukey J. W., An algorithm for machine calculationof complex Fourier series, Math. Computation, 19 (1965), 297-301.

[3] Haar A., Zur Theorie der orthogonalen Funktionensysteme, Math.Ann., 69 (1910), 331-371.

[4] Herley C.; Kovacevic J.; Ramchandran K.; Vetterli M., Arbitraryorthogonal tilings of the time-frequency plane (Int. Symp. on Time-Frequency and Time-Scale Analysis, 11-14), Victoria, BC 1992.

[5] Herley C.; Kovacevic J.; Ramchandran K.; Vetterli M., Tilingsof the time-frequency plane: construction of arbitrary orthogonalbases and fast tiling algorithms, IEEE Trans. Signal Processing, 41(1993), 3341-3359.

[6] Herley C.; Xiong Z.; Ramchandran K.; Orchard M. T., Joint space-frequency segmentation using balanced wavelet packet trees for leastcost image representation, IEEE Trans. Image Processing, 6 (1997),1213-1230.

[7] Ramchandran K.; Vetterli M., Best wavelet packet bases in a rate-distortion sense, IEEE Trans. Image Processing, 2 (1993), 160-173.

[8] Thiele C. M.; Villemoes L. F., A Fast Algorithm for Adapted Time–Frequency Tilings, Applied and Computational Harmonic Analysis,3 (1996), 91-99.

[9] Walsh J. L., A Closed Set of Normal Orthogonal Functions, Amer.J. Math., 45 (1923), 5-24.

[10] Wickerhauser M. V., Lectures on wavelet packet algorithms,Preprint, Department of Mathematics, Washington University,1991.


Generalized tilings with height functions ∗

Olivier Bodini Matthieu Latapy

Abstract

In this paper, we introduce a generalization of a class of tilingswhich appear in the literature: the tilings over which a heightfunction can be defined (for example, the famous tilings of poly-ominoes with dominoes). We show that many properties of thesetilings can be seen as the consequences of properties of the gen-eralized tilings we introduce. In particular, we show that anytiling problem which can be modelized in our generalized frame-work has the following properties: the tilability of a region canbe constructively decided in polynomial time, the number of con-nected components in the undirected flip-accessibility graph canbe determined, and the directed flip-accessibility graph induces adistributive lattice structure. Finally, we give a few examples ofknown tiling problems which can be viewed as particular cases ofthe new notions we introduce.

2000 Mathematics Subject Classification: Primary: 05B45; Secondary:52C20Keywords and phrases: Tilings, Height Functions, Tilability, Distribu-tive Lattices, Random Sampling, Potentials, Flows.

1 Preliminaries

Given a finite set of elementary shapes, called tiles, a tiling of a givenregion is a set of translated tiles such that the union of the tiles cov-ers exactly the region, and such that there is no overlapping betweenany tiles. See for example Figure 1 for a tiling of a polyomino (set of

∗This work arose from ideas develop by the first author while he was completinghis Ph.D. studies at the Mathematics Department of the University of P. and M.Curie at Paris.

47

48 Olivier Bodini and Matthieu Latapy

squares on a two-dimensional grid) with dominoes (1×2 and 2×1 rect-angles). Tilings are widely used in physics to modelize natural objectsand phenomena. For example, quasicrystals are modelized by Penrosetilings [4] and dimers on a lattice are modelized by domino tilings [10].Tilings appeared in computer science with the famous undecidability ofthe question of whether the plane is tilable or not using a given finite setof tiles [2]. Since then, many studies appeared concerning these objects,which are also strongly related to many important combinatorial prob-lems [11]. A local transformation is often defined over tilings. This

Figure 1: From left to right: the two possible tiles (called dominoes),a polyomino (i.e. a set of squares) to tile, and a possible tiling of thepolyomino with dominoes.

transformation, called flip, is a local rearrangement of some tiles whichmakes it possible to obtain a new tiling from a given one. One thendefines the (undirected) flip-accessibility graph of the tilings of a regionR, denoted by AR, as follows: the vertices of AR are all the tilings ofR, and t, t′ is an (undirected) edge of AR if and only if there is a flipbetween t and t′. See Figure 2 for an example. The flip notion is akey element for the generation and enumeration of the tilings of a givenregion, and for many algorithmical questions. For example, we will seein the following that the structure of AR may give a way to samplerandomly a tiling of R with the uniform distribution, which is crucialfor physicists. This notion is also a key element to study the entropy ofthe physical objects [13], and to examine some of their properties likefrozen areas, weaknesses, and others [6].

On some classes of tilings which can be drawn on a regular grid, itis possible to define a height function which associates an integer to anynode of the grid (it is called the height of the node). For example, onecan define such a function over domino tilings as follows. As alreadynoticed, a domino tiling can be drawn on a two dimensional squaregrid. We can draw the squares of the grid in black and white like on

Generalized tilings with height functions 49

Figure 2: From left to right: the flip operation over dominoes, andtwo examples of tilings which can be obtained from the one shown inFigure 1 by one flip. In these tilings, we shaded the tiles which movedduring the flip.

a chessboard. Let us consider a polyomino P and a domino tiling T ofP , and let us distinguish a particular node p on the boundary of P , saythe one with smaller coordinates. We say that p is of height 0, and thatthe height of any other node p′ of P is computed as follows: initializea counter to zero, and go from p to p′ using a path composed only ofedges of dominoes in T , increasing the counter when the square on theright is black and decreasing it when the square is white. The height ofp′ is the value of the counter when one reaches p′. One can prove thatthis definition is consistent. This can be used as the height function fordomino tilings [18]. See Figure 3 for an example.

These height functions make it possible to define AR, the directedflip-accessibility graph of the tilings of a region R: the vertices of AR arethe tilings of R and there is a directed edge (t, t′) if and only if t can betransformed into t′ by a flip which decreases the sum of the heights of allthe vertices (a flip always changes the height function). See Figure 3 foran example with domino tilings. The generalized tilings we introduce inthis paper are based on these height functions, and most of our resultsare induced by them.

These notions of height functions are related to classical notions offlow theory in graphs. Let G = (V,E) be a directed graph. A flow onG is a map from E into C (actually, we will only use flows with valuesin Z). Given two vertices v and v′ of G, a travel from s to s′ is a set ofedges of G such that, if one forgets their orientations, then one obtainsa path from s to s′. Given a flow C, the flux of C on the travel T is

FT (C) =!

e∈T+

C(e) −!

e∈T−

C(e)

where T+ is the set of oriented edges of T which are traveled in the


0 1 0 1

−1

1 0 10 0

−1

0

23 2

−2

0 1 0 1

−1

1 0 10 0

−1

0

22

0 1 0 1

−1

1 0 10 0

−1

0

2

0 1 0 1

−1

1 0 10 0

−1

0

−2

0 1 0 1

−1

1 0 10 0

−1

0

−1

−1 −1

−1

−2

−2

2

Figure 3: The directed flip-accessibility graph of the tilings of a poly-omino by dominoes. The height of each node of the polyomino is shownfor each tiling. The set of all the tilings of this polyomino is ordered bythe flip relation directed with respect to the height functions.

right direction when one goes from s to s′, and T− is the set of orientededges traveled in the reverse direction. One can easily notice that theflux is additive by concatenation of travels: if T1 and T2 are two travelssuch that the ending point of T1 is equal to the starting point of T2,then FT1·T2(C) = FT1(C)+FT2(C). See [1] for more details about flowtheory in graphs. Then we notice that a height function is a flux.

Since there is no circuit in the graph AR (there exists no nonemptysequence of directed flips which transforms a tiling into itself), it inducesan order relation over all the tilings of R: t ≤ t′ if and only if t′ canbe obtained from t by a sequence of (directed) flips. In Section 3, wewill study AR under the order theory point of view, and we will meetsome special classes of orders, which we introduce now. A lattice isan order L such that any two elements x and y of L have a greatestlower bound, called the infimum of x and y and denoted by x ∧ y, anda lowest greater bound, called the supremum of x and y and denoted


by x ∨ y. The infimum of x and y is nothing but the greatest elementamong the ones which are lower than both x and y. The supremumis defined dually. A lattice L is distributive if for all x, y and z in L,x∨(y∧z) = (x∨y)∧(x∨z) and x∧(y∨z) = (x∧y)∨(x∧z). For example,it is known that the flip-accessibility graph of the domino tilings of apolyomino without holes is always a distributive lattice [17]. Therefore,this is the case of the flip-accessibility graph shown in Figure 3 (noticethat the maximal element of the order is at the bottom, and the minimalone is at the top of the diagram since we used the discrete dynamicalmodels convention: the flips go from top to bottom). Lattices (andespecially distributive lattices) are strongly structured sets. Their studyis an important part of order theory, and many results about themexist. In particular, various codings and algorithms are known aboutlattices and distributive lattices. For example, there exists a genericalgorithm to sample randomly an element of any distributive latticewith the uniform distribution [16]. For more details about orders andlattices, we refer to [8].

Finally, let us introduce a useful notation about graphs. Given adirected graph G = (V,E), the undirected graph G = (V ,E) is thegraph obtained from G by removing the orientations of the edges. Inother words, V = V , and E is the set of undirected edges v, v′ suchthat (v, v′) ∈ E. We will also call G the undirected version of G. Noticethat this is consistent with our definitions of AR and AR.

In this paper, we introduce a generalization of tilings on which aheight function can be defined, and show how some known results maybe understood in this more general context. All along this paper, likewe did in the present section, we will use the tilings with dominoes as areference to illustrate our definitions and results. We used this uniqueexample because it is very famous and simple, and permits to give clearfigures. We emphasize however on the fact that our definitions andresults are much more general, as explained in the last section of thepaper.

2 Generalized tilings.

In this section, we give all the definitions of the generalized notions weintroduce, starting from the objects we tile to the notions of tilings,height functions, and flips. The first definitions are very general, there-fore we will only consider some classes of the obtained objects, in order


to make the more specific notions (mainly height functions and flips)relevant in this context. However, the general objects introduced maybe useful in other cases.

Let G be a simple directed graph (G has no multiple edges, no loops,and if (v, v′) is an edge then (v′, v) can not be an edge). We considera set Θ of elementary circuits of G, which we will call cells. Then, apolycell is any set of cells in Θ. Given a polycell P , we call the edges ofcells in P the edges of P , and their vertices the vertices of P . A polycellP is k-regular if and only if there exists an integer k such that each cellof P is a circuit of length k. Given a polycell P , we fix an arbitrarilynot empty partial subgraph of P that we denote by ∂P . We call it theboundary of P . Notice that, it is not a topological definition. In thefollowing, a polycell will be always considered with its fixed boundary,that is to say (P, ∂P ). A polycell P is full if ∂P is connected.

Given an edge e of P which is not on the boundary, we call the setof all the cells in P which have e in common a tile. A set of edges ofP\∂P such that the associated tiles constitute a partition of the cellsof P is called a tiling Q. An edge in Q is by definition a tiling edge. Apolycell P which admits at least a tiling Q is tilable. See Figure 4 andFigure 5 for some examples. Notice that if we distinguish exactly oneedge of each cell of a polycell P , in such a way that none of them is onthe boundary of P , then the distinguished edges can be viewed as thetiling edges of a tiling of P . Indeed, each edge induces a tile (the set ofcells which have this edge in common), and each cell is in exactly onetile.

Figure 4: From left to right: a polycell P (the boundary ∂P beingthe partial subgraph constituted by the edges belonging to a uniqueelementary circuit and Θ being the set of all the elementary circuits),the three tiles of P , and a tiling of P represented by its tiling edges (thedotted edges). This polycell is full, tilable, and is not k-regular for anyk. Notice that there are two tiles composed of two cells, and anotherone composed of three cells. Notice also that the tiling given in thisfigure is the only possible one.


Figure 5: Left: a 4-regular polycell P (the cells in Θ are the circuits oflength 4), the boundary of which is composed of those edges belongingto a unique elementary circuit. Right: a tiling of P represented by itstiling edges (the dotted edges). Notice that this figure is very similar toFigure 1.

Let P be a k-regular tilable polycell and Q be a tiling of P . Weassociate to Q a flow CQ on Θ (seen as a graph):

CQ(e) =

!1− k if the edge e is a tiling edge of Q1 otherwise.

For each cell c, we define Tc as the travel which contains exactly theedges of c (in other words, it consists in turning around c). Notice thatthe flux of CQ on the travel Tc is always null: FTc(CQ) = 0 since eachcell contains exactly a tiling edge, valued 1− k, and k − 1 other edges,valued 1. Moreover, for each edge e ∈ ∂P , we have CQ(e) = 1 sincefrom the definition e cannot be a tiling edge.

Let us consider a polycell P and a flow C on the edges of P . If forevery closed travel T (i.e. a cycle when one forgets the orientation ofeach edge) in P we have FT (C) = 0, then the flow C is called a tension.A polycell P is contractible if the fact that FTc(C) = 0 for every cell cimplies that C is a tension. Since the converse is always true, we thenhave that C is a contractible if the following is true: C is a tensiononly if for every cell c, FTc(C) = 0. Notice that if P is a contractiblek-regular polycell and Q is a tiling of P , then the flow CQ is a tension,since for every cell c, FTc(CQ) = 0.

Now, if we (arbitrarily) distinguish a vertex ν on the boundary ofP , we can associate to the tension CQ a potential ϕQ, defined over thevertices of P :


• ϕQ(ν) = 0.

• for all vertices x and y of P , ϕQ(y) − ϕQ(x) = FTx,y(CQ) whereTx,y is a travel from x to y.

The distinguished vertex is needed otherwise ϕQ would only be definedexcept for a constant, but one can choose any vertex on the boundary.Notice that this potential can be viewed as a height function associatedto Q, and we will see that it indeed plays this role in the following.Therefore, we will call the potential ϕQ the height function of Q. SeeFigure 6 for an example.

1 1 1 1

1

11

11

1 1 −31−3 −3

−3 1

0 1 0 1 0

−1

0

2 3 2

1 0 1 0

−1

1 1 1 1

1

Figure 6: From left to right: a tiling Q of a polycell (represented by itstiling edges, the dotted ones), the tension CQ and the height function(or potential) ϕQ it induces. Again, this figure may be compared toFigure 3 (topmost tiling).

We now have all the main notions we need about tilings of poly-cells, including height functions, except the notion of flips. In order tointroduce it, we need to prove the following:

Theorem 2.0.1 Let P be a k-regular contractible polycell. There is abijection between the tilings of P and the tensions C on P which verify:

• for every edge e in ∂P , C(e) = 1,

• and for every edge e of P , C(e) ∈ 1− k, 1.

Proof: For every tiling Q of P , we have defined above a flow CQ

which verifies the property in the claim, and such that for every cell c,FTc(CQ) = 0. Since P is contractible, this last point implies that CQ

is a tension. Conversely, let us consider a tension C which satisfies thehypotheses. Since each cell is of length k, and since C(e) ∈ 1 − k, 1,the fact that FTc(C) = 0 implies that each cell has exactly one negativeedge. These negative edges can be considered as the tiling edges of atiling of P , which ends the proof. !

Given a k-regular contractible polycell P defined over a graph G, thistheorem allows us to identify a tiling Q and the associated tension CQ.


This makes it possible to define the notion of flip as follows. Supposethere is a vertex x in P which is not on the boundary and such thatits height, with respect to the height function of Q, is greater than theheight of each of its neighbors in G. We will call such a vertex a maximalvertex. The neighbors of x in G have a smaller height than x, thereforethe outgoing edges of x in G are tiling edges of Q and the incomingedges of x in G are not. Let us consider function CQ′ defined as follows:

CQ′(e) =

⎧⎨

⎩

1 if e is an outgoing edge of x1− k if e is an incoming edge of xCQ(e) else.

Each cell c which contains x contains exactly one outgoing edge of x andone incoming edge of x, therefore we still have FTc(CQ′) = 0. Therefore,CQ′ is a tension, and so it induces from Theorem 2.0.1 a tiling Q′. Wesay that Q′ is obtained from Q by a flip around x, or simply by a flip.Notice that Q′ can also be defined as the tiling associated to the heightfunction obtained from the one of Q by decreasing the height of x by k,and without changing anything else. This corresponds to what happenswith classical tilings (see for example [17]). See Figure 7 for an example.

00 0

22

1 1

3

0 1 0 1 0

−1−1

1 1

1 1

11

1

1

1−3

−3

1−3

−3 −3 −3

1 11 1

111 1

1

1

11

1

1−3

1 1

1

1−3

1

1 1

1

1111

00 0

22

1 1

0 1 0 1 0

−1−1−1

Figure 7: A flip which transforms a tiling Q of a polycell P into anothertiling Q′ of P . From left to right, the flip is represented between thetilings, then between the associated tensions, and finally between theassociated height functions.

We now can define and study AP , the (directed) flip-accessibilitygraph of the tilings of P : AP = (VP , EP ) is the directed graph whereVP is the set of all the tilings of P and (Q,Q′) is an edge in EP if


Q can be transformed into Q′ by a flip. We will also study the undi-rected flip-accessibility graph AP . The properties of these graphs arecrucial for many questions about tilings, like enumeration, generationand sampling.

3 Structure of the flip-accessibility graph.

Let us consider a k-regular contractible polycell P and a tiling Q of P .Let h be the minimal value among the heights of all the vertices withrespect to the height function of Q. If Q is such that all the vertices ofheight h are on the boundary of P , then it is said to be a maximal tiling.For a given P , we denote by TmaxP the set of the maximal tilings ofP . We will see that these tilings play a particular role in the graphAP . In particular, we will give an explicit relation between them andthe number of connected components of AP . Recall that we defined theminimal vertices of Q as the vertices which have a height least than theheight of each of their neighbors, with respect to the height function ofQ (they are local minimums).

Lemma 3.0.2 Let P be a k-regular tilable contractible polycell (P isnot necessarily full). There exists a maximal tiling Q of P .

Proof: Let V be the set of vertices of P , and let Q be a tiling of Psuch that for every tiling Q′ of P , we have:

!

x∈VϕQ(x) ≥

!

x∈VϕQ′(x).

We will prove that Q is a maximal tiling. Suppose there is a minimalvertex xm which is not on the boundary. Therefore, one can transformQ into Q′ by a flip around xm. Then

"x∈V ϕQ′(x) =

"x∈V ϕQ(x)+k,

which is in contradiction with the hypothesis. !

Lemma 3.0.3 For every tiling Q of a k-regular contractible polycell P ,there exists a unique tiling in TmaxP reachable from Q by a sequence offlips.

Proof: It is clear that at least one tiling in TmaxP can be reached fromQ by a sequence of flips, since the flip operation decreases the sum of theheights, and since we know from the proof of Lemma 3.0.2 that a tilingsuch that this sum is maximal is always in TmaxP . We now have toprove that the tiling in TmaxP we obtain does not depend on the order


in which we flip around the successive minimal vertices. Since makinga flip around a minimal vertex x is nothing but increasing its height byk and keeping the other values, if we have two minimal vertices x andx′ then it is equivalent to make first the flip around x and after the fliparound x′ or the converse. !

Lemma 3.0.4 Let P be a k-regular contractible and tilable polycell. Atiling Q in TmaxP is totally determined by the values of ϕQ on ∂P .

Proof: The proof is by induction over the number of cells in P . Letx be a minimal vertex for ϕQ in ∂P . For all ingoing edges e of x,CQ(e) = 1−k (otherwise ϕ(x) would not be minimal). Therefore, theseedges can be considered as tiling edges, and determine some tiles of atiling Q of P . Iterating this process, one finally obtains Q. See Figure 8for an example. !

Theorem 3.0.5 Let P be a k-regular contractible and tilable polycell.The number of connected components in AP is equal to the cardinal ofTmaxP .

Proof: Immediate from Lemma 3.0.3. !This theorem is very general and can explain many results which

appeared in previous papers. We obtain for example the followingcorollary, which generalizes the one saying that any domino tiling ofa polyomino can be transformed into any other one by a sequence offlips [3].

Corollary 3.0.6 Let P be a full k-regular contractible and tilable poly-cell. There is a unique element in TmaxP , which implies that AP isconnected.

Proof: Since ∂P is connected, the heights of the vertices in ∂P aretotally determined by the orientation of the edges of ∂P and do notdepend on any tiling Q. Therefore, from Lemma 3.0.4, there is a uniquetiling in TmaxP . !

As a consequence, if P is a full tilable and contractible polycell,the height of a vertex x on the boundary of P is independent of theconsidered tiling. In the case of full polyominoes, this restriction ofϕQ to the boundary of P is called height on the boundary [9] and hasbeen introduced in [18]. Notice also that the proof of Lemma 3.0.4gives an algorithm to build the unique maximal tiling of any k-regularcontractible and tilable full polycell P , since the height function on the


boundary of P can be computed without knowing any tiling of P . SeeAlgorithm 1 and Figure 8. This algorithm gives in polynomial timea tiling of P if it is tilable. It can also be used to decide whether Pis tilable or not. Therefore, it generalizes the result of Thurston [18]saying that it can be decided in polynomial time if a given polyominois tilable with dominoes.

1 1 1 1

1

1

1 1 1 1

1

1

1

1

0

1 0 1

1 0 1

−1

0

0

0

1

1

2

1 1 1 1

1

1

0

1 0 1

1 0 1

−1

0

1 1 1 1

0

1 1 1 1

1

1

0

1 0 1 0

1 0 1

−1

0

1 1 1 1

0 1

2

1

−3

1

1

1

1

−1

1

1

1

1

−1

−3

−3−3

−31 1

1

1

1 1

−2 −1 −2

Figure 8: An example of execution of Algorithm 1. The polycell we wantto tile is drawn on the top. Its boundary is composed of the verticeswhich belong to at most three edges. Below, from left to right, we showthe result of each iteration of the algorithm (computation of the tensionon the first line, and of the height function on the second line). In thisexample, the first iteration of the algorithm gives one vertical tile, andthe second (and last) iteration gives four horizontal tiles.

With these results, we obtained much information concerning a cen-tral question of tilings: the connectivity of the undirected flip-accessibilitygraph. We did not only give a condition under which this graph is con-nected, but we also gave a relation between the number of its connectedcomponents and some special tilings. We will now deepen the study ofthe structure induced by the flip relation by studying the directed flip-accessibility graph, and in particular the partial order it induces overthe tilings: t ≤ t′ if and only if t′ can be obtained from t by a sequenceof (directed) flips.


Algorithm 1 Computation of the maximal tiling of a full k-regular contractible polycell.

Input: A full k-regular contractible polycell P , its boundary ∂Pand a distinguished vertex ν on this boundary.

Output: An array tension on integers indexed by the edges ofP and another one height indexed by the vertices of P .The first gives the tension associated to the maximaltiling, and the second gives its height function.

beginP ′ ← P ;height[ν]← 0;for each edge e = (v, v′) on the boundary of P ′ do

tension[e]← 1;

for each vertex v in ∂P ′ doCompute height[v] using the values in tension;

repeatfor each vertex v in ∂P ′ which has the maximal heightamong the heights of all the vertices in ∂P ′ do

for each incoming edge e of v dotension[e]← 1− k;for each edge e′ in a cell containing e do

tension[e′]← 1;

for each edge e = (v, v′) such that tension[e] hasnewly been computed do

Compute height[v] and height[v′] using the valuesin tension;

Remove in P ′ the cells which contain a negative edge;Compute the boundary of P ′: it is composed of all thevertices of P ′ which have a computed height;

until P ′ is empty ;

end


Lemma 3.0.7 Let Q and Q′ be two tilings in the same connected com-ponent of AP of a polycell P . By definition, we put:

X = := x such that ϕQ(x) = ϕQ′(x),

we take xm ∈ X= such that:

max(ϕQ(xm),ϕQ′(xm)) = maxϕQ(x),ϕQ′(x);x ∈ X=

(in other words, xm is the maximum among the vertices x where ϕQ(x) =ϕQ′(x)), We can suppose that ϕQ(xm) > ϕQ′(xm) - else we exchange Qand Q′ -, then the potential ϕ defined by :ϕ(x) := ϕQ(xm)− k, when x := xmand by ϕ(x) := ϕQ(x), otherwiseis associated to a tiling of P .

Proof: Let us consider any cell which contains xm. Therefore, it con-tains an incoming edge (xp, xm) and an outgoing edge (xm, xs) of xm.We will prove that ϕQ(xp) = ϕQ(xm)−1 and ϕQ(xs) = ϕQ(xm)−k+1,which will prove the claim since it proves that xm is a maximal vertexfor ϕQ and so ϕ defines a tiling of P (which is therefore obtained fromQ by a flip around xm).

The couple (ϕQ(xp),ϕQ(xs)) can have three values: (ϕQ(xm) −1,ϕQ(xm) +1), (ϕQ(xm)−1,ϕQ(xm)−k+1), or (ϕQ(xm)+k−1,ϕQ(xm)+1). But, if ϕQ(xs) = ϕQ(xm)+1 then ϕQ′(xs) = ϕQ(xm)+1 (xs /∈ X =),and so ϕQ′(xm) = ϕQ(xm) + k (xm ∈ X =), which is in contradictionwith ϕQ(xm) > ϕQ′(xm). Therefore, (ϕQ(xp),ϕQ(xs)) must be equal to(ϕQ(xm)− 1,ϕQ(xm)− k+1) for every cell which contain xm, which iswhat we needed to prove. !

Let us now consider two tilings Q and Q′ of a k-regular contractiblepolycell P . Let us define max(ϕQ,ϕQ′) as the height function such thatits value at each vertex is the maximal between the values of ϕQ andϕQ′ at this vertex. Let us define min(ϕQ,ϕQ′) dually. Then, we havethe following result:

Lemma 3.0.8 Given two tilings Q and Q′ in the same connected com-ponent of AP for a k-regular contractible polycell P , max(ϕQ,ϕQ′) andmin(ϕQ,ϕQ′) are the height functions of tilings of P .

Proof: We can see that max(ϕQ,ϕQ′) is the height function of a tilingof P by iterating Lemma 3.0.7:

!x

""ϕQ(x)− ϕQ′(x)"" can be decreased

until we reach max(ϕQ,ϕQ′). The proof for min(ϕQ,ϕQ′) is symmetric.!


Theorem 3.0.9 If P is a k-regular contractible polycell, then each con-nected component of AP induces a distributive lattice structure over thetilings of P .

Proof: Given two tilings Q and Q′ in the same connected compo-nent of AP , let us define the following binary operations: ϕQ ∧ ϕQ′ =min(ϕQ,ϕQ′) and ϕQ ∨ ϕQ′ = max(ϕQ,ϕQ′). It is clear from the pre-vious results that this defines the infimum and supremum of Q and Q′.To show that the obtained lattice is distributive, it suffices now to verifythat these relations are distributive together. !

As already discussed, this last theorem gives much information onthe structure of the flip-accessibility graphs of tilings of polycells. Italso gives the possibility to use in the context of tilings the numerousresults known about distributive lattices, in particular the generic ran-dom sampling algorithm described in [16].

To finish this section, we give another proof of Theorem 3.0.9 usingonly discrete dynamical models notions. This proof is very simple andhas the advantage of putting two combinatorial object in a relationwhich may help understanding them. However, the reader not interestedin discrete dynamical models may skip the end of this section.

An Edge Firing Game (EFG) is defined by a connected undirectedgraph G with a distinguished vertex ν, and an orientation O of G. Inother words, O = G. We then consider the set of obtainable orientationswhen we iterate the following rule: if a vertex v = ν only has incom-ing edges (it is a sink) then one can reverse all these edges. This setof orientations is ordered by the reflexive and transitive closure of theevolution rule, and it is proved in [15] that it is a distributive lattice.We will show that the set of tilings of any k-regular contractible poly-cell P is isomorphic to configuration space of an EFG, which impliesTheorem 3.0.9.

Let us consider a k-regular contractible polycell P defined over agraph G. Let G′ be the sub-graph of G which contains exactly thevertices and edges in P plus a new vertex ν and a edge (v, ν) for every vin ∂P . This vertex will be the distinguished vertex of our EFG. Let usnow consider the height function ϕQ of a tiling Q of P , and let us definethe orientation π(Q) of G′ as follows: the edges involving ν are directedtowards ν, and each other undirected edge v, v′ in G′ is directed fromv to v′ in π(Q) if ϕQ(v′) > ϕQ(v). Then, the maximal vertices of Q areexactly the ones which have only incoming edges in π(Q), and applyingthe EFG rule to a vertex of π(Q) is clearly equivalent to making a flip


around this vertex in Q. Moreover, one can never apply the EFG ruleto a vertex in ∂P , since it always has an outgoing edge to ν, whichcan never be reversed. Finally, the configuration space of the EFG isisomorphic to the connected component of AP which contains Q, whichproves Theorem 3.0.9 again. An example is given in Figure 9.

Figure 9: The configuration space of the EFG obtained from Figure 3(the distinguished vertex ν is not represented: there is an additionaloutgoing edge from each vertex on the boundary to ν). The two ordersare isomorphic.

4 Some applications.

In this section, we present some examples which appear in the literature,and we show how these tiling problems can be seen as special cases of k-regular contractible polycells tilings. We therefore obtain as corollariessome known results about these problems, as well as some new results.

4.1 Polycell drawn on the plane or the sphere.

Let us consider a set of vertices V and a set Θ of elementary (undirected)cycles of length k, with vertices in V , such that any couple of cycles in Θhave at most one edge in common. Now let us consider the undirected


graph G = (V,E) such that e is an edge of G if and only if it is an edgeof a cycle in Θ. Moreover, let us restrict ourselves to the case where Gis a planar graph which can be drawn in such a way that no cycle of Θ isdrawn inside another one. G is 2-dual-colorable if one can color in blackand white each bounded face in such a way that two faces which have anedge in common have different colors. See for example Figure 10. The

Figure 10: Two examples of graphs which satisfy all the properties givenin the text. The leftmost is composed of cycles of length 3 and has ahole. The rightmost one is composed of cycles of length 4.

Figure 11: A tiling of each of the objects shown in Figure 10, obtainedusing the polycells formalism.

fact that G has the properties above, including being 2-dual-colorable,makes it possible to encode tilings with bifaces (the tiles are two adjacentfaces) as tilings of polycells. This includes tilings with dominoes, andtilings with calissons. Following Thurston [18], let us define an orientedversion of G as follows: the edges which constitute the white cyclesboundaries are directed to travel the cycle in the clockwise orientation,and the edges which constitute the black cycles boundaries are directedcounterclockwise. If for every closed travel (its origin and its extremitycoincide) on the boundary of polycell P we have FT (C) = 0 where Cis a flow such that C(e) = 1 for every e ∈ ∂P , then we say that P hasa balanced boundary. One can verify that a polycell with a balancedboundary defined in this way is always contractible. Therefore, ourresults can be applied, which generalizes some results of Chaboud [5]and Thurston [18].


4.2 Rhombus tiling in higher dimension.

Let us consider the canonical basis e1, . . . , ed of the d-dimensionalaffine space Rd, and let us define ed+1 =

!di=1 ei. For every α between

1 and d+1, let us define the zonotope Zαd,d as the following set of points:

Zαd,d = x ∈ Rd such that x =

d+1"

i=1,i =α

λiei, with − 1 ≤ λi ≤ 1.

In other words, the Zαd,d is the zonotope defined by all the vectors ei

except the α-th. We are interested in the tilability of a given solid Swhen the set of allowed tiles is Zα

d,d, 1 ≤ α ≤ d+1. These tilings arecalled codimension one rhombus tilings, and they are very important asa physical model of quasicristals [7]. If d = 2, they are nothing but thetilings of regions of the plane with three parallelograms which tile anhexagon, which have been widely studied. See Figure 12 for an examplein dimension 2, and Figure 13 for an example in dimension 3. In orderto encode this problem by a problem over polycells, let us consider thedirected graph G with vertices in Zd and such that e = (x, y) is an edgeif and only if y = x+ej for an integer j between 1 and d or y = x−ed+1.We will call diagonal edges the edges which correspond to the secondcase. This graph can be viewed as a d-dimensional directed grid towhich we add a diagonal edge in the reverse direction, at each point ofthe grid. An example in dimension 3 is given in Figure 14. Each edge

Figure 12: If one forgets the orientations and removes the dotted edges,then the rightmost object is a classical codimension one rhombus tilingof a part of the plane (d = 2). From the polycells point of view, theleftmost object represents the underlying graph G, the middle objectrepresents a polycell P (the boundary of which is the set of the edgeswhich belong to only one cell), and the rightmost object represents atiling of P (the dotted edges are the tiling edges).

is in a one-to-one correspondence with a copy of a Zαd,d translated by an

integer vector, namely the one of which it is the diagonal edge. The setΘ of the cells we will consider is the set of all the circuits of length d+1


Figure 13: A codimension one rhombus tiling with d = 3 (first line,rightmost object). It is composed of four different three dimensionaltiles, and the first line shows how it can be constructing by addingsuccessive tiles. The second line shows the position of each tile withrespect to the cube.

Figure 14: Top: the 3-dimensional grid is obtained by a concatenationof cubes with reverse diagonal edges, like this one. Bottom: the cells inΘ. Each tile is composed of six such cells, since each edge belongs toexactly six cells.

which contain exactly one diagonal edge. Therefore, each edge belongsto d! cells, and so the tiles will be themselves composed of d! cells. SeeFigure 14 for an example in dimension 3. Given a polycell P definedover Θ, we define ∂P as the partial subgraph composed by the edgesof P which do not belong to d! circuits of P . First notice that a fullpolycell defined over G is always contractible. Therefore, our previousresults can be applied, which generalizes some results presented in [7]and [12]. We also generalize some results about the 2-dimensional case,which has been widely studied.


5 Conclusion and Perspectives.

In conclusion, we gave in this paper a generalized framework to studysome tiling problems over which a height function can be defined. Thisincludes the famous tilings of polyominoes with dominoes, as well asvarious other classes, like codimension one rhombus tilings, tilings withholes, tilings on torus, on spheres, three-dimensional tilings, and otherswe did not detail here. We gave some results on our generalized tilingswhich made it possible to obtain a large set of known results as corollar-ies, as well as to obtain new results on tiling problems which appear inthe scientific literature. Many other problems may exist which can bemodelized in the general framework we have introduced, and we hopethat this paper will help understanding them.

Many tiling problems, however, do not lead to the definition of anyheight function. The key element to make such a function exist is thepresence of a strong underlying structure (the k-regularity of the poly-cell, for example). Some important tiling problems (for example tilingsof zonotopes) do not have this property, and so we can not apply ourresults in this context. Some of these problems do not have the strongproperties we obtained on the tilings of k-regular contractible polycells,but may be included in our framework, since our basic definitions ofpolycells and tilings being very general. This would lead to general re-sults on more complex polycells, for example polycells which are notk-regular, or with cells which have more than one edge in common.

Acknowledgement

The authors thank Frederic Chavanon for useful comments on pre-liminary versions, which deeply improved the manuscript quality.

Olivier BodiniLIRMM,University of Montpellier 2,161, rue Ada, 34392 MontpellierCedex 5, [email protected]

Matthieu LatapyLIAFA,University of Paris 7,2, place Jussieu, 75005 Paris,[email protected]

References

[1] Ahura R.K., Network Flows Theory, Algorithms and Applications,Prentice Hall (1976).


[2] Berger R., The indecidability of the domino problem, Mem. ofAmer. Math. Soc., 66 (1966).

[3] Beauquier D.; Nivat M.; Remila E.; Robson J.M., Tiling figuresof the plane with two bars, a horizontal and a vertical one, Com-putational Geometry, 5 (1995), 1-25.

[4] Caspar D.L.D; Fontano E., Five-fold symmetry in crystalline qua-sicrystal lattices, Proc. Natl. Acad. Sci.USA, 93 (1996), 14271-14278.

[5] Chaboud T., Domino Tiling in planar graphs with regular andbipartite dual, TCS, 159 (1996), 137-142.

[6] Cohn H.; Propp J.; Shor P., Random domino tilings and the arcticcircle theorem, (2001) Preprint.

[7] Destainville N.; Mosseri R.; Bailly F., Configurational entropy ofcodimension-one tilings and directed membranes, Journal of Sta-tistical Physics, 87, (1997), 697-713.

[8] Davey B.A.; Priestley H.A., Introduction to lattices and orders,Cambridge University Press, (1990).

[9] Fournier J.C., Tiling pictures of the plane with dominoes, DiscreteMathematics, 156 (1997), 313-320.

[10] Kenyon R., The planar dimer model with boundary: a survey,AMS-CRM Monogr., (2000), 307-328.

[11] Latapy M., Generalized integer partitions, tilings of zonotopes andlattices, preprint.

[12] Linde J.; Moore C.; Nordahl M.G., An n-dimensional general-ization of the rhombus tiling, Proceedings of DM-CCG’01 (2001),23-42.

[13] Lagarias J.C.; Romano D.S., A polyomino tiling ploblem ofThurston and its configuration entropy, Journal of CombinatorialTheory, 63 (1993), 338-358.

[14] Moore C.; Robson J.M., Hard tiling problems with simple tiles,preprint to appear in Discrete Mathematics.


[15] Propp J., Lattice structure of orientations of graphs, preprint(1993).

[16] Propp J., Generating random elements of finite distributive lat-tices, Electronic Journal of Combinatorics, 4 (1998).

[17] Remila E., The lattice structure of the set of domino tilings of apolygon, to appear.

[18] Thurston W.P., Conway’s tiling groups, Amer. Math. Monthly, 97(1990), 757-773.



Osvaldo Osuna-Castro 1

Abstract

In this work we prove that given an autonomous Lagrangian Lon a closed manifold M , if an Anosov energy level k can bereparametrized to make it of contact type, then k > c0(L), thecritical value of L associated with the abelian covering.

2000 Mathematics Subject Classification: 37D40, 53D25.Keywords and phrases: Anosov flows, asymptotic cycle, contact type

.seulavlacitircsenaM,slevel

1 Introduction

Let M be a closed connected manifold, TM its tangent bundle. Anautonomous Lagrangian is a smooth function, L : TM → R convexand superlinear. This means that L restricted to each TxM has positivedefinite Hessian and for some Riemannian metric we have

lim|v|→∞

L(x, v)

|v| = ∞,

uniformly on x. Since M is compact, the Euler-Lagrange equation de-fines a complete flow ϕt on TM . Recall that the energy E : TM → Ris defined by

E(x, v) :=∂L

∂v(x, v)v − L(x, v).

Since L is autonomous, E is a first integral of the flow ϕt. Let us set

e := maxx∈ME(x, 0) = −minx∈ML(x, 0).

1Ph.D. Student, CIMAT, Guanajuato, Gto., Mexico. Supported by a scholarshipfrom the CONACYT.

69

70 Osvaldo Osuna-Castro

Note that the energy level E−1k projects onto the manifold M if andonly if k ≥ e.

We shall denote by L:TM → T ∗M the Legendre transform which isdefined by (x, v) → ∂L

∂v (x, v). Our hypotheses on L assure that L is adiffeomorphism. Let H : T ∗M → R be the Hamiltonian associated toL:

H(x, v) := maxv∈TxMpv − L(x, v).If θ denotes the canonical 1-form on T ∗M , then the Euler-Lagrange flowof L can be also obtained as the Hamiltonian flow of E with respect tothe symplectic form on TM given by −L∗dθ thus, if X denotes the vectorfield associated with the Euler-Lagrange flow then iXL∗dθ = −dE. Inother words, the energy function satisfies E = H L, so that energylevels for L are sent to level sets of H.Definition: An energy level Σ = H−1k is of contact type if thereexists a 1-form λ on Σ such that dλ = ω(= −dθ) and λ(X) = 0 onevery point of Σ.

An Anosov energy level, is a regular energy level E−1k on whichthe flow ϕt is an Anosov flow.

In [6] G. Paternain shows that if an Anosov energy level k on asurface can be reparametrized to make it of contact type then k > c0(L)the critical value of L associated with the abelian covering. Our goal inthis note is to generalize this result, we shall prove the following:

Theorem A. Given an autonomous Lagrangian L on a closed manifoldM with dim M ≥ 2 , If an Anosov energy level k can be reparametrizedto make it of contact type then k > c0(L).

This completes the previous result.

2 Preliminaries and proofs

Let M(L) be the set of probabilities on the Borel σ-algebra on TMthat have compact support and are invariant under the flow ϕt. LetH1(M,R) be the first real homology group of M . Given a closed one-form ω on M and ρ ∈ H1(M,R), let ⟨[w], ρ⟩ denote the integral of ω onany closed curve in the homology class ρ. If µ ∈ M(L), its homology isdefined as the unique ρ(µ) ∈ H1(M,R) such that

⟨[w], ρ(µ)⟩ =!

TMωdµ,

Anosov energy levels 71

for all closed 1-form on M . The integral on the right-hand side is withrespect to µ with ω considered as the function ω : TM → R.

Let µ be a ϕt-invariant probability supported on the energy levelΣ = E−1k. The Schwartzman’s asymptotic cycle S(µ) ∈ H1(Σ,R) ofµ is defined by

⟨[Ω],S(µ)⟩ =!

ΣΩ(X)dµ,

for every closed 1-form Ω on Σ, where X is the Lagrangian field on Σ.The homology ρ(µ) of the measure µ is the projection of its asymptoticcycle by π∗ : H1(Σ,R) → H1(M,R).

Recall that the L-action of an absolutely continuous curve γ : [a, b] →M is defined by

AL(γ) :=! b

aL(γ(t), γ(t))dt.

Given two points x1, x2 ∈ M and some T > 0 denoted by C(x1, x2) theset of absolutely continuous curves γ : [a, b] → M with γ(0) = x1 andγ(T ) = x2. For each k ∈ R, we define

Φk(x1, x2; T ) := infAL+k(γ) | γ ∈ C(x1, x2) .

The action potential Φk : M × M → R ∪∞ of L is defined by

Φk(x1, x2) := infT>0Φk(x1, x2; T ).

Definition (Mane): The critical value of L is the real number

c = c(L) := infk | Φk(x, x) > −∞ for some x ∈ M.

Note that if k > c(L) actually Φk(x, x) > −∞ for all x ∈ M . SinceL is convex and superlinear, and M is compact, such a number exists.We can also consider the critical value of the lift of the Lagrangian Lto a covering of the compact manifold M . Suppose that p : N → Mis a covering space and consider the Lagrangian L : TN → R givenby L := Ldp, for each k ∈ R we can define an action potential Φk inN ×N just as above and similarly we obtain a critical value c(L) for L.It can be easily checked that if N1 and N2 are coverings of M such thatN1 covers N2 then

c(L1) ≤ c(L2),

where L1 and L2 denote the lifts of the Lagrangian L to N1 and N2

respectively.


Among all possible coverings of M there are two distinguished ones;the universal covering which we shall denote by !M , and the abelian cov-ering which we shall denote by M . The latter is defined as the coveringof M whose fundamental group is the kernel of the Hurewicz homo-morphis π1(M) → H1(M,R) these coverings give rise to the criticalvalues

cu(L) := c("L) and ca(L) = c0(L) := c(L)

where "L and L denote the lifts of the Lagrangian L to !M and M respec-tively. Therefore we have cu(L) ≤ c0(L), but in general the inequalitymay be strict as it was shown in [5].

2.1 Contact and Anosov energy levels

We begin by introducing some concepts related to Euler-Lagrange flowrestricted on energy levels.

Definition: An energy level Σ = H−1k is of contact type if thereexists a 1-form λ on Σ such that dλ = ω(= −dθ) and λ(X) = 0 onevery point of Σ.

Equivalently, if there exists a vector field Y based on Σ, such thatthe Lie derivative LY ω = ω. The correspondence is given by iY ω = λ.The vector field Y must be tranverse to Σ because if it is tangent to Σone has that λ(X) = ω(Y, X) = dH(Y ) = 0.

Lemma 2.2.1 The set k ∈ R | H−1k is of contact type is open.

Proof: Suppose that Σ = H−1k is of contact type, then k is a regularpoint of H, for otherwise the Hamiltonian flow contains a singularityon Σ and that violates the condition λ(X) = 0. If λ is a contact formfor λ, since dλ = ω then λ = pdx|Σ + τ , where τ is a closed 1-form onΣ. We can extend λ as follows. Let π : U → Σ be the projection of anopen neighbourhood U of Σ onto Σ. Let λ := pdx + π∗(τ) then dλ = ωand for m near k dλ|H−1m = 0.

The following criterion for contact type appears in [2]

Proposition 2.2.2 If L is a convex Lagrangian then an energy levelE−1k is of contact type if and only if

#TM (L+k)dµ > 0 for any invari-

ant measure µ supported E−1k with zero asymptotic cycle S(µ) = 0.

We shall need the following result:


Lemma 2.2.3 Suppose M a closed connected manifold with dim M ≥ 2and M = T 2. If k > e then π∗ : H1(E−1k,R)→ H1(M,R) is anisomorphism.

Proof: Since k > e and dim M ≥ 2 then the energy level E−1k is iso-morphic to the unit tangent bundle of M with the canonical projection.Using the Gysin exact sequence of the circle bundle π : E−1k → Mone can show that (see [3], lemma 1.45) the lemma follows if M is ori-entable.

If M is not orientable and dim M ≥ 3, using the exact homotopysequence of the circle bundle π : E−1k → M :

0 = π1(Sn−1) → π1(E−1k) π∗→ π1(M) → π0(Sn−1) = 0,

thus we obtain that π∗ : π1(E−1k) → π1(M) is an isomorphism, whichin turn implies that π∗ : H1(E−1k,R)→ H1(M,R) is a isomorphism.In the case that M is not orientable and dim M = 2, the proof is aminor modification of the above arguments.

An Anosov energy level, is a regular energy level E−1k on whichthe flow ϕt is an Anosov flow. In [1] was shown

Proposition 2.2.4 If the energy level k is Anosov, then

k > cu(L).

In [5] G. Paternain and M. Paternain gave examples of Anosov en-ergy levels k with k < c0(L) on surface of genus greater or equal thantwo. These examples give a negative answer to a question raised byMane.

2.3 Proof of theorem A

Now we shall prove the theorem A, for this we use the next result ofPaternain [4] and following his ideas we shall prove this result

Proposition 2.3.1 If cu(L) < k < c0(L), there exists an invariantmeasure µ supported in the energy level k, such that ρ(µ) = 0 and

!

E−1k(L + k)dµ ≤ 0.


Proof of theorem A: It follows from a result of Margulis that the energylevels of T 2 does not support Anosov flows thus in the case of T 2 thetheorem is valid trivially. Therefore we can assume that M = T 2. Nowas the flow is Anosov, by the proposition 2.2.4 we have that k > cu(L).But if the energy level k ∈ (cu(L), c0(L)) then applying the proposition2.3.1, there exists an invariant measure µ such that ρ(µ) = 0 and

!

E−1k(L + k)dµ ≤ 0.

therefore the lemma 2.2.3 and proposition 2.2.2 implies that, the energylevel k is not of contact type. Finally by lemma 2.2.1 the energy k =c0(L) cannot be of contact type then, we must have that k > c0(L).

Proof of proposition 2.3.1 : Since k < c0(L) = ca(L) there exists T > 0and an absolutely continuous closed curve γ : [0, T ] → M homologousto zero such that

AL+k(γ) < 0.(1)

For n ≥ 1, let us denote by γn : [0, nT ] → M the curve γ wrappedup n times. Since k > cu(L), by (1) γn cannot be homotopic to zero.Let p : "M → M the covering projection and take y such that p(y) =γ(0) = γ(T ), and let #γn : [0, nT ] → "M be the unique lift of γn with#γn(0) = y. As k > cu(L) for each n there exists a solution xn(t) ofEuler-Lagrange with energy k and some Tn > 0 such that xn(0) = yand xn(Tn) = #γn(nT ).

Let µn denote the probability measure in TM uniformly distributedalong pxn|[0,Tn] and take µ a point of accumulation of µn, this measureµ has the required properties of the proposition 2.3.1.

Osvaldo Osuna CastroCIMAT,A. Postal 402,3600 Guanajuato, Gto., [email protected]

References

[1] Contreras, G.; Iturriaga, I.; Paternain, G. P.; Paternain, M., La-grangian graphs, minimizing measure and Manes critical values,Geom. Func. Anal., 8 (1998), 788-809.


[2] McDuff, D., Applications of convex integration to symplectic andcontact geometry, Ann. Inst. Fourier, 37 (1987), 107-133.

[3] Paternain, G. P., Geodesic flows, Birkhauser Boston Inc, Boston,MA, 1999.

[4] Paternain, G. P., Hyperbolic dynamics of Euler-Lagrange flowson prescribed energy levels, Seminaire de Therie spectrale etGeometrie, 1996-97, Grenoble, 15 (1998).

[5] Paternain, G. P.; Paternain, M., Critical values of autonomousLagrangian systems, Comment. Math. Helvet., 72 (1997), 481-499.

[6] Paternain, G. P., On the regularity of the Anosov splitting fortwisted geodesic flows, Math Res. Lett., 4 (1998), 871-888.

MORFISMOS, Comunicaciones Estudiantiles del Departamento de Matema-ticas del CINVESTAV, se termino de imprimir en el mes de junio de 2003 enel taller de reproduccion del mismo departamento localizado en Av. IPN 2508,Col. San Pedro Zacatenco, Mexico, D.F. 07300. El tiraje en papel opalinaimportada de 36 kilogramos de 34 × 25.5 cm consta de 500 ejemplares en pastatintoreto color verde.

Apoyo tecnico: Omar Hernandez Orozco.

Contenido

The Lagrange approach to constrained Markov control processes: a survey andextension of results

Raquiel R. Lopez-Martınez and Onesimo Hernandez-Lerma . . . . . . . . . . . . . . . 1

noiccelesaledamelborpleyaicneucerf-opmeitnesatercsidsenoicatneserpeRde frecuencias

uitinaMunaetsraCierdnAnilA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Generalized tilings with height functions

Olivier Bodini and Matthieu Latapy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47


Osvaldo Osuna-Castro . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

morfismos, vol 7, no 1, 2003

Documents

artculo publicado

direccion decorreo electronico

in the same or similar

in addition

departamentode matematicas

apartado postal

revista morfismosde

medio electronico