1

Lecture 1

Nicolo Michelusi

• Textbooks

• Grades

• Midterm

• Project

• Homeworks

I. INTRODUCTION

Many of you have taken ECE547. In ECE547, the goal is often to analyze a system (e.g., a

queuing system), or to analyze a protocol or an algorithm (e.g., a priority queuing discipline,

Bellman-Ford algorithm, CSMA-CD).

While performance analysis is crucial, for many research problems you may not be given an

algorithm or protocol to begin with. In these cases, you are free to design your own.

How do you design an algorithm/protocol that, in a certain sense, is best?

Example: transmission in a wireless channel with fading.

Due to mobility and fading, the channel between transmitter (base station) and receiver (mobile

terminal) changes over time. What should the transmission power at the base station be when

the fading state is g?

Different strategies:

1) Fixed power

January 4, 2019 DRAFT

2

2) Variable power to maintain target rate

3) Some other technique

What is best? The answer depends on the formulation of a proper objective function and on its

solution –> Optimization problem

Example: what if the base station is transmitting to multiple mobile terminals simultaneously?

And what if there are multiple base stations or femto-cells?

Also in this case, we can come up with different strategies to allocate users to base stations

(e.g. closest base station, strongest gain), and to serve users at a specific base station (e.g., user

with the strongest channel, or TDMA). Determining which scheme is best involves formulating

a proper optimization problem (e.g., minimize power, maximize rate, maximize fairness, etc.)

and solving it to find the optimal solution.

II. OPTIMIZATION

In the first part of the course, we will study convex optimization, a useful tool to answer many

of these questions.

We will follow this process:

1) Build a system model, e.g.: channel g(t), fading in i.i.d. fashion over time t according to

some distribution P (g); achievable rate r(t) = log2(1 + p(t)g(t)/σ2), where p(t) is the

transmission power (to be optimized) and σ2 is the noise power.

2) Identify the objectives to optimize, e.g.: minimize average power E[p(t)]; maximize trans-

mission rate E[r(t)]; or a trade-off between the two.

3) Identify the constraints, e.g.: maximum power constraint; minimum rate requirement

4) Formulate an optimization problem, e.g.: let {gk, k = 1, . . . , N} be a set of N channel

gains and qk = P(g(t) = gk). Let pk be the power level used when g(t) = gk. Then,

minp

∑k

qk log2(1 + pkgk/σ2) (1)

January 4, 2019 DRAFT

3

s.t.∑k

qkpk ≤ Pmax (2)

Is this the only possible formulation? No! We might formulate different optimization prob-

lems, depending on our goal and the application.

5) Find the solution to this problem, e.g.: the optimal power allocation

p∗k = f(gk, Pmax, q)

In many cases, a closed-form solution is not available; in these cases, we may be happy

with an iterative algorithm that approximately finds the optimal solution.

Be it direct or iterative solutions, often:

• we prefer online adaptive over offline

• estimating model parameters can be time and computationally consuming

• we prefer schemes that are robust to inaccuracies

• we prefer schemes that adapt to changes over time

• we prefer distributed over centralized solutions (single point of failure, lack of scalability)

An online distributed solution can potentially be turned into a protocol that can be used in real

systems.

III. CONVEX PROBLEMS

In this class, we will study the so-called convex problems:

• Control variables are real numbers, not integers

• Objective function and constraints are convex

Convex optimization problems are important because:

• They model many (not all) interesting and practical problems

• There are well-studied and rigorous techniques (with performance guarantees) to solve these

problems

• Some of the techniques developed for convex problems can also be used in non-convex

problems (however, without performance guarantees)

It is not straightforward to formulate a problem into a convex form. More of an art than a

science.

The goal of the first part of this class is:

January 4, 2019 DRAFT

4

• understand the characteristics of convex problems and learn common approaches to solve

them

• gain some experience in formulating research problems into convex problems and solve

them

• understand the issues and challenges to develop practical protocols based on convex opti-

mization.

The class will cover:

• Fundamentals of theory: convexity, duality, convergence

• Applications in communications and networking:

– PHY: detection, compression

– MAC: power control, scheduling, random access

– Transport: congestion control, TCP

– Routing: multipath routing, network flow

– Cross-layer design for wireless

• Applications in power systems:

– Demand-side management

– Renewable integration

We will use these applications to illustrate how convex optimization problems are used in

practice.

IV. OTHER EXAMPLES

Many problems can be framed in this way:

Regression or parameter estimation: Find the best linear approximation that is closest to the

observed input output relationship (xk, yk)

January 4, 2019 DRAFT

5

Hypothesis testing and detection: how to decide on the original symbol, so that it has the least

error

Routing: each link has a certain bandwidth, delay, channel gain, error probability. Find the

best route from A to B: maximize throughput; or minimize delay; or minimize error rate; etc.

Scheduling subject to delay: you have to send a bunch of packets through a channel; each

packet has a certain load, delay requirement. How do you schedule transmissions to minimize

the packet loss? What if the channel is subject to fading? What if the packet arrival process is

random?

January 4, 2019 DRAFT

6

Placement in sensor networks: how do you place cluster heads in a WSN to optimize coverage;

or minimize average transmission power?

Route optimization in UAV aided sensor networks: an UAV is used to collect data from a

wireless sensor network over a wide area; what is the optimal route that should be followed by

the UAV for this task?

V. SEQUENTIAL OPTIMIZATION UNDER UNCERTAINTY

In the second part of the class, we will learn optimization under uncertainty. Uncertainty arises

in many systems:

• Channel conditions vary over time

• Users come and go

• Renewal supply from wind and sun change over time

• etc.

Additionally, the environment we operate in may change as a result of actions we perform in

the system:

January 4, 2019 DRAFT

7

• A mobile user may transmit, causing interference to other users; as a result of this inter-

ference, these users might fail their transmission and may need to retransmit their packets.

Dilemma: shall the mobile user transmit now or defer its transmission?

• A mobile user wants to transmit a packet within a deadline over a fading wireless channel,

with minimum energy. Dilemma: shall the mobile user wait for the fading channel to improve

(so as to save energy), or transmit now at higher energy cost (so as to reduce the backlog of

untransmitted bits)? In this case, the action transmit has an effect, since it results in energy

consumption and smaller backlog; the action remains idle has the opposite effect.

In these cases, we need techniques beyond convex optimization. In this course, we will

learn about Markov decision processes and we will learn dynamic programming algorithms

to determine the optimal action for each state of the environment. In order to do that, we will

formulate an optimization problem, where the optimization variable is the policy which maps

states to actions.

We will also introduce a major problem in dynamic programming algorithms: the curse

of dimensionality, due to the often high-dimensional state and action spaces (for instance, in

January 4, 2019 DRAFT

8

a wireless sensor networks, each user has its own data queue, so the state is the vector of

queue states for each user). In this case, we will introduce approximate dynamic programming

algorithms to solve this problem.

Finally, in most practical systems, the statistical properties (e.g., probability distributions of

events, rewards, etc.) are unknown and need to be estimated in an online fashion. In these cases,

dynamic programming algorithms cannot be readily applied, since they require knowledge of the

statistical properties of the system we are operating in. We will introduce reinforcement learning

algorithms to address these problems.

January 4, 2019 DRAFT