IN DEGREE PROJECT COMPUTER SCIENCE AND ENGINEERING,SECOND CYCLE, 30 CREDITS

, STOCKHOLM SWEDEN 2019

Optimizing LTE test traffic using search- and expert algorithms

WILLIAM SCHRÖDER

KTH ROYAL INSTITUTE OF TECHNOLOGYSCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE

Optimizing LTE test traffic using search- and expert algorithms

William Schröder

Master thesis in computer science (30 credits) at the school of computer science and engineering

Royal Institute of Technology 2019-05-26

Supervisor at CSC was Anders Holst Examiner was Örjan Ekeberg

Abstract This thesis investigates whether artificial intelligence techniques can replace

the optimization process that is currently performed by experts for LTE/4G

stability tests at Ericsson. The investigation covers the usage of search

algorithms and expert systems and the thesis then implements a hybrid

algorithm. This implemented solution is then tested upon three different

architectures to judge how well it optimizes tests without causing further

problems. The conclusion is that the implemented solution optimizes tests

sufficiently but does so using only mobile users. To reflect reality better, static

users would also need to be included in the optimization process.

Sammanfattning Syftet med det här exjobbet är att undersöka huruvida tekniker baserade på

artificiell intelligens skulle kunna ersätta de optimeringssteg som för

närvarande görs av experter under stabilitetstester för LTE/4G vid Ericsson.

Bakgrundsundersökningen täcker sökalgoritmer och expertsystem och en

hybridalgoritm har sedan implementerats. Ett experiment har därefter

genomförts på tre olika testfall för att utvärdera den implementerade

algoritmens förmåga att optimera test utan att introducera nya problem i

testet. Slutsatsen är att optimeringen har tillräcklig effekt men att detta endast

görs genom att omfördela mobila användare. Statiska användare skulle också

behöva inkluderas i algoritmen för att göra dess optimeringar mer realistiska.

Table of Contents 1 Introduction............................................................................................................................. 1

1.1 Problem description ..................................................................................................... 1

1.2 Objective ............................................................................................................................ 2

1.3 Scope ................................................................................................................................... 2

1.4 Evaluation ......................................................................................................................... 2

1.5 Outline ................................................................................................................................ 3

1.6 Terminology ..................................................................................................................... 3

2 Background .............................................................................................................................. 5

2.1 LTE technology ............................................................................................................... 5

2.2 Lab environment ............................................................................................................ 6

2.2.1 Architecture.............................................................................................................. 7

2.2.2 UE assignment ......................................................................................................... 8

2.2.3 Test execution and evaluation .......................................................................... 8

2.2.4 Optimal test configuration .............................................................................. 11

3 Related methods ................................................................................................................. 13

3.1 Optimization problems ............................................................................................ 13

3.2 Local search algorithms ........................................................................................... 14

3.2.1 Heat based search ............................................................................................... 16

3.2.2 Multi-start search................................................................................................ 17

3.2.3 Genetic algorithm ................................................................................................ 19

3.2.4 Multi-neighbor search....................................................................................... 20

3.2.5 Alternating neighborhood search ................................................................ 21

3.3 Knowledge based systems ...................................................................................... 21

3.3.1 Rule based .............................................................................................................. 22

3.3.2 Fuzzy logic .............................................................................................................. 22

3.3.3 Training based...................................................................................................... 24

3.3.4 Analysis ................................................................................................................... 24

3.4 Hybrid algorithms ...................................................................................................... 24

3.4.1 Interpolation with expert system ................................................................ 25

3.4.2 Optimizing binary search with expert system........................................ 25

3.5 Flow networks ............................................................................................................. 25

4 Method .................................................................................................................................... 27

4.1 Problem formalization ............................................................................................. 27

4.2 Node traffic discovery ............................................................................................... 28

4.3 Cell traffic optimization ........................................................................................... 28

4.4 Solution strategy ......................................................................................................... 29

4.5 Node level formula analysis ................................................................................... 30

4.6 Cell level formula analysis ...................................................................................... 31

4.7 Node level traffic redistribution ........................................................................... 31

4.8 Cell level traffic redistribution .............................................................................. 32

4.8.1 The flow network approach ........................................................................... 32

4.8.2 Multiple bad cells ................................................................................................ 35

4.8.3 Handover types priority ................................................................................... 36

4.8.4 Net flow between nodes ................................................................................... 36

5 Experiment............................................................................................................................ 37

5.1 Node level formula analysis ................................................................................... 37

5.2 Cell level formula analysis ...................................................................................... 37

5.3 LTE test configuration .............................................................................................. 38

5.3.1 Formulae used ...................................................................................................... 39

5.3.2 Test cases................................................................................................................ 39

5.4 Evaluation metrics ..................................................................................................... 41

5.5 Results ............................................................................................................................. 41

6 Discussion.............................................................................................................................. 44

6.1 Analysis ........................................................................................................................... 44

6.2 Criticism.......................................................................................................................... 45

6.3 Alternative solutions ................................................................................................. 45

7 Conclusion ............................................................................................................................. 47

8 References ............................................................................................................................. 48

1 - Introduction

1

1 Introduction The theme of this thesis is search algorithms and expert systems. The purpose

is to investigate whether algorithms in these fields can be deployed in time

constrained environments with no previous knowledge and still perform with

adequate results.

1.1 Problem description The problem of this thesis is provided by the LTE stability team of Ericsson.

Their objective is to ensure that the specifications of the radio base stations

sold by Ericsson to its customers pass the stability requirements. To

accomplish this, they regularly perform tests in their networks with simulated

roaming users. The requirements tested form a subset of the many test

demands that exists across Ericsson and are for the stability team focused on

analyzing how the base stations react to multiple users switching to and from

its coverage zones. For each test, they specify the hardware of the base

stations being tested as well as the initial number of users throughout the

network. The goal is to have as many handovers between the base stations as

possible without them showing any indications of instability in the test

results.

Because tests take considerable time to run (around 3 hours), the

configuration steps are very delicate. As the team configure the tests to be on

the borderline of what the base stations can handle, there is an overwhelming

risk of there being too much, causing instability. In these cases, the testers

must analyze the test results and try to come up with a new configuration that

would remove the errors, while reducing the number of handovers as little as

possible. This process is far from trivial and takes up a lot of the testers’ time.

Thus, there is a need to have an artificial intelligence help the testers

determine how to redistribute the simulated users in order for the next test to

pass the requirements. Ideally, the algorithm should be able to run

autonomously overnight, allowing the system multiple iterations of

improvements to reach the optimal configuration.

The considerable amount of time it takes to run a test is also the source of the

uniqueness of this problem, which is that data points required for the

optimization are expensive to produce. Thus, the contributions of this thesis

1 - Introduction

2

are: a literature study on optimization methods with this constraint in mind; a

theoretical suggestion of how an iterative algorithm for solving the described

problem can be designed; and an implementation and evaluation of the

algorithm for a single step in this iteration. Simulated tests, like the ones used

in this thesis, contributes to a sustainable environment by avoiding the need

for some hardware tests, reducing power consumption. By automating these

tests, this thesis also makes a contribution in sustainability. Ethical concerns

are not an issue as the work is done in automation which affects nothing but the tests ran of one tester on simulated equipment.

1.2 Objective The goal of the thesis is to investigate what algorithms can be used to solve

the test optimization problem and to do a comparison between these. The

primary objective is to find an algorithm which reliably suggest improvements

to a test configuration given its latest results. This algorithm must provide

improvement suggestions that accommodate all the experts’ requirements

while minimizing the reduction of handovers. There can be no requirement on

training data as no test history is being kept of how different configurations

yield what results. A secondary objective has also been set out. The suggested

algorithm should be able to run multiple iterations and use results from more

than one set of test results during optimization.

The specific problem statement is

Can an algorithm that suggests optimizations to LTE mobility tests be

constructed? If so, could such algorithm be designed to converge using

test results of multiple iterations?

1.3 Scope Only one iteration of the proposed algorithm is tested in the test networks.

Also, out of the discussed theoretical alternatives only one implementation

could be tested for availability reasons.

1.4 Evaluation From Ericsson’s perspective, evaluating the effectiveness of the

implementation concerns measuring the usefulness of the test configuration

suggestions it provides to the testers. Ideally, these would be evaluated by

1 - Introduction

3

comparing the results of a test conducted with the initial configuration with

one using the configuration suggested by the proposed algorithm. However,

for practical purposes this was not possible and thus the evaluation will

instead be based on a function provided by the experts. This leads to the

following evaluation formulation:

The algorithm is given an initial test configuration and the

corresponding test results to find a better configuration. The suggested

changes are then analyzed using a function formulated by the experts to

deduce how likely they are to lead to improved test results. If the

evaluation suggests the changes should yield improved results, the

algorithm has been successful.

To ensure the algorithm can handle a variety of test scenarios, the algorithm is

tested upon three typical test configurations. These are:

1. A small test (few routes), testing the algorithm’s operating capacity

when the setting is constrained.

2. A medium sized test (standard routes), aimed to symbolize the

general usage case.

3. A large test (many routes), challenging the algorithm to make

optimal assignments when the solution space is larger.

1.5 Outline The thesis first covers the details of LTE as a background to understanding the

tests conducted. This knowledge is then used to model the underlying

optimization problem and to divide it into sub-problems. Each such sub-

problem is then investigated individually with a literature study to determine

which algorithms to pursue in the implementation phase. Next, parts of the

algorithm are implemented and evaluated using the previously described

strategy.

1.6 Terminology This section is meant as a quick summary of the terms used in this report.

Each of the terms is more thoroughly described later in the report.

LTE: The technical term for 4G technology.

1 - Introduction

4

Node: Also referred to as ENB this is the abbreviation for radio base

stations used in 4G/LTE and is represented as n.

Cell: A subcomponent of a node which provides coverage in a certain

direction. It can be visualized as a circular arc originating from the node

where users can connect to the node. Cells are represented as c. A cell c

is considered to belong to a node n if c is a subcomponent of n.

Route: A path describing user movement between two cells, thus

performing a specific handover between these cells. Routes have a

handover type and are represented as r. A route r is considered to

belong to a cell c if r has an endpoint in c. Similarly, r is considered to

belong to a node n if there is a cell c such that r belongs to c and c

belongs to n.

Handover type: A protocol the cells use when performing a handover

of users to another cell. Can be either S1, X2, intra or irat.

Traffic: Also referred to as load or less accurately as UEs or users;

traffic is an intensity measurement of how many handovers that are

performed during a given time period. Traffic is represented as t and is

a property of routes, in which case tr represents the amount of traffic on

route r.

Formula: A specific test result field, presented in success- or failure

rate. An example would be node level handover rejects due to high load.

Formulae are represented as f. Formula measurement denotes the

value a specific formula has for a given test. Formula filter is the

function which returns a Boolean value as to whether a formula

measurement is within the tolerance range or not.

2 - Background

5

2 Background As a prerequisite to analyzing the optimization problem that is the basis for

the implementation, this section covers the basics of LTE technology as well as

a more detailed description of the test networks in the stability team.

2.1 LTE technology The test network that is being utilized in this thesis is a simulation of the LTE

network, sometimes referred to as 4G. At the time of writing (March 2017)

this is the most recent cellular communication technology. It consists of these

components:

• The User Equipment (UE).

• The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN).

• The Evolved Packet Core (EPC).

See figure 2-1 below for a diagram of the subsystems.

Figure 2-1: LTE network architecture

The UEs corresponds to user held devices, such as cell phones or tablets, that

are connected to the LTE network. They each contain a sim card which

identifies them across the network. There are various categories and

properties of the UEs but for this thesis they can simply be considered as

users.

The E-UTRAN is the subsystem in closest proximity to the UEs and provide the

access points to the LTE infrastructure, known as eNodeBs (ENB), henceforth

abbreviated as nodes. These are the LTE correspondents to what in GSM is

known as the Radio Base Stations (RBS) and act as links between the UEs and

the packet core (EPC). The nodes are also responsible for controlling the

operations of its connected UEs which among other things includes handling

handovers to other nodes when the UEs leave its area of operations. To

accomplish these objectives, two communication interfaces are available to

2 - Background

6

the nodes, (Poole, n.d.). The S1 interface is used to communicate with the EPC.

The main form of data transferred through this interface is user data

transmitted between nodes and the EPC, i.e. network data. The other available

interface, X2, is more commonly used to communicate handover information

with other nodes directly as the UEs leave one node for another. See figure 2-2

for an overview of the E-UTRAN subsystem.

Figure 2-2: LTE access network

Additionally, there is a third interface, known as irat, used for communicating

handover information to nodes of other mobile technologies should UEs move

from an LTE node to, say a WCDMA NB (the 3G correspondent of a node).

There are also instances where handovers within the LTE network would run

over the S1 interface rather than through X2, which is normally employed.

This means that the handovers are communicated through the EPC layer

rather than through direct communication between two nodes.

2.2 Lab environment This section focuses on the test network environment at 4G Stability. There

are a variety of configuration settings for these networks, the ones of interest

to this thesis are:

• The test network architecture which describes the hardware and the

handover types of the UEs.

• The amount of traffic on each component of the architecture.

The rest of this chapter covers these configurations in order as well as a

description of how tests are executed and evaluated.

2 - Background

7

2.2.1 Architecture

The architecture of the test network is the configuration of the actual

hardware. This primarily constitutes of the nodes used and what kind of

limitations these have. In a real network, each node has one or more coverage

zones which provides network access in the direction they are facing. Each

such coverage area is referred to as a cell. From a top-down view, each cell

can be thought of as a circular arc extended from the node, indicating its

coverage zone. Such a view can be seen if figure 2-3 where the black dots

represent the nodes and the yellow arcs represents the cells. There are also

red arrows in this figure, which are explained in the next couple of

paragraphs.

Figure 2-3: Test network architecture

A UE positioned in any of the cells is connected to the node of that cell. For the

special case where a UE is positioned in the overlapping section of two cells it

is connected to one of the two. In addition, UEs can be either static, stationary

UEs, or moving in which case they are referred to as mobile UEs. This report

focuses only on mobile UEs since the tests of interest to the stability team

concerns handovers, which can only be achieved with mobile UEs.

A real-world mobile UE, such as a car travelling along a highway, would follow

a certain path and thus entering and leaving the coverage zones of different

cells repeatedly. In the test networks, this concept is simplified to only

consider how frequently UEs perform handovers between any two cells in the

architecture instead of considering the complete path of the UEs. The reason

for this simplification is that the objective of the test of the stability team is to

2 - Background

8

provide specifications of the node capabilities individually, rather than

analyzing mobile UEs’ impact on the complete architecture. For this reason,

test UEs are defined using routes which represent a specific handover

between two cells of the same or different nodes. figure 2-3 depicts routes as

red arrows between two cells, indicating all mobile UEs on this route would

perform a handover between these two cells. In the same figure, there are two

routes, defined as H1 and H2. These routes have a significant difference as H1

is an inter-route between different nodes while H2 is an intra-route between

two cells of the same node. Recall from the

LTE technology section that inter-routes can be conducted using three

different protocols, S1 (via the EPC interface) and X2 (direct node to node

handover) or irat (to a node of another mobile technology). Handovers using

intra-routes do not have any associated handover type.

2.2.2 UE assignment

With the infrastructure fully configured using the architecture, the next step is

to put traffic into the test. Traffic is represented by a set amount of mobile

UEs that operate on a certain route, thus performing the handover stated by

this route. Thus, by increasing traffic on a route, the connected nodes will

have a higher load since they need to deal with more handovers. Ideally, the

load on each node in the test should be as close as possible to the hardware

limitations without exceeding them.

2.2.3 Test execution and evaluation

After a test has been configured with architecture and traffic, it is executed in

one of the test networks. During the test run, traffic is simulated with various

intensity between low and high peaks. The evaluation measurements for the

test is accessible as soon as the test starts, but at this stage everything is zero.

The longer the test runs the more accurate these measurements are. Tests are

always deterministic meaning two tests that run with the same configuration

produces the same results.

2 - Background

9

Evaluating the results of a test is done by examining the results of

measurements known as formulae1. The full evaluation results of a test

involve hundreds of different formulae, each describing how well the test

went with respect to some parameters, whereof only a handful are relevant to

the stability team. Formulae are separated into two categories: node- and cell

level. An example would be ‘handovers rejects due to high load’ which is a

node-level formula depicting the number of attempted handovers that failed

due to the node already operating at peak capacity. A formula can also be

defined on the cell level, then referred to as cell-level formula. All future

references to formulae refer to the subset of formulae that are relevant for the

stability team.

The value of each formula, referred to as formula measurement is presented

in percent. Node-level formulae indicate rates of failure, meaning the goal is to

be as low as possible, while cell-level formulae indicate rates of success and

should thus be maximized. Furthermore, the formulae have a tolerance range

where architecture components performed adequately. The Boolean value for

a specific formula is referred to as formula filter and is denoted by g where

g(f) is True if the formulae measurement of f is within the tolerance range of f.

Since the formulae of interest varies from test to test, including the tolerance

ranges, these parameters must be specified as input to each test.

As the traffic intensities vary throughout the test, so does the evaluation

measurements. For this reason, formulae measurements are sampled at

specific time intervals of 15 minutes or more. Figure 2-4 depicts such an

interval sampling during the time period 08:15 to 17:15 using a sampling

interval of 60 minutes. Two test runs are visible in the graph, one which was

started before 08.15 and ended around the graph dip at 11.15. Another test

was then started which finished around 17.15.

1 The name formula originates from the fact that evaluation measurements are based on some sort of calculation (i.e. successful attempts divided by total attempts) and thus it is said that the evaluation measurements is the result of a formula.

2 - Background

10

Figure 2-4: Formulae measurement sampling

Experts of the stability team have analyzed the relationship between load and

formulae measurements and determined that the behavior of this function is

too complex in nature to be used for optimization in the general test case.

Thus, the formulae measurements of each test must be analyzed separately.

The experts have also provided some general information as to how load

relates to formulae measurements. For the node level, formulae

measurements are expected to have a roughly linear relationship with traffic.

On the cell level however, reducing load on a cell marginally affects the

formulae measurements of that cell until a certain point where there is a rapid

improvement in formulae measurements. A schematic illustration is depicted

in Figure 2-5.

2 - Background

11

Figure 2-5: Schematic illustration of formulae measurements relationship with load.

2.2.4 Optimal test configuration

With the details of the test environment covered, the question is now how the

thesis’ goal should be reached, i.e. how to construct the optimal test. There are

two factors contributing to evaluating the quality of the tests:

• The formulae filters, indicating to what extent the hardware passed the

set-up requirements. The more formulae passed, the better the results.

• The amount of traffic, indicating how much pressure the hardware was

under. The more traffic, the more useful the results.

Between these factors, it is more important to pass the configured formulae

filters than having much load on the node. Thus, the goal when optimizing the

test can be expressed as “configuring the test with maximum possible traffic

amount on each node such that the test still passes the formulae filters”.

Should one or more nodes or cells not pass any of the corresponding formulae

filters then those are considered bad nodes or bad cells. In such cases, the

bad nodes and/or cells needs to have their traffic reduced. The most straight

forward way of performing this traffic reduction would be to remove static

UEs from the bad node/cell. However, since the stability team is interested in

Form

ula

mea

surm

ent

Load

Schematic illustration of the load-formula measurement

relationship

Cell level Node level

2 - Background

12

testing how the nodes’ capacity in handling handovers, the traffic reduction

has to be made with mobile UEs. Recall from the Architecture section that

mobile UEs are added using routes between different cells, and thus that there

is a conflict of interest between the connected cells as to what the reduction

should be.

Thus, configuring an optimal test is a matter of finding traffic assignments

to routes in such a way that all formulae filters for each node and cell in

the architecture pass while traffic is maximized on each node. This is the

formulation for the underlying optimization problem which is analyzed

further in the Method section.

3 - Related methods

13

3 Related methods This section covers the basis of the algorithms designed later in the report. Its

primary function is to act as a resource for the reader to further understand

these underlying concepts before proceeding with the report. The main points

of this chapter are basics on optimization problems, local search algorithms,

knowledge based systems and flow networks.

3.1 Optimization problems Optimization problems can be defined as the following: “for a given problem

with a space of solutions, find the best such solution”. The solution space

consists of a set of unknown variables where a valid assignment of these

variables is a potential solution to the problem. To determine the best

solution, an objective function that depends on these variables is used. The

goal when solving the optimization problem is to either minimize or maximize

the objective function by configuring the variables.

Moreover, the range of the variables in the solution space determines the

characterization of the optimization problem. Firstly, the solution space can

be discrete or continuous and can also be subject to constraints in what is

then defined as a constrained optimization problem. Constraints can either

be defined as a relationship- or range limitations of the variables in the

solution space. Additionally, the optimization problem can have either zero,

one or multiple objectives. While one objective function is common, multiple

objective functions are also possible (e.g. finding the cheapest and quickest

flight available). In these cases, one normally must specify a weight for the

different objectives (e.g. finding a cheap flight that should also be quick if

possible) for the objective requirement(s) to be interpreted correctly. The

case of zero objectives is also interesting, in this case the goal is simply to find

a legal variable assignment. An evaluation function is used to evaluate how

well a solution matches the objectives.

When faced with an optimization problem, there are two different approaches

to finding a solution. The most obvious one, a solver, solves an instance of the

problem. The other approach, a modeler, attempts to find relationships

between constraints and variables to analyze the problem in the general case.

3 - Related methods

14

To illustrate the difference between these methods, consider the problem of

finding the quickest flight path between London and New York. The solution

of this problem varies from day to day due to the air corridor allocation,

weather etc. These factors limit the options and thus they are constraints in

this constrained optimization problem. To find a solution for a certain time

where the air corridors and weather conditions are given, a solver would be

employed to find the solution. If, however, the goal is to analyze the general

form of the problem (i.e. without knowing the air corridors and weather

conditions) then a modeler would be used to identify relationships between

the constraints and the solution.

3.2 Local search algorithms A local search algorithm is a method used to solve optimization problems. In

addition, it is often used to improve existing solutions that other algorithms

have found. A good example of this is the TSP problem. Assume that another

algorithm has already found a solution to a certain instance and the question

is now if it is possible to start from the solution and try to find improvements.

A local search algorithm can accomplish this by exploring two of the paths

found in the previous algorithm and investigate whether the route would be

faster by switching these two routes. This process is referred to as 2-opt and

is quick to run since one iteration of the algorithm takes constant time. This

optimization has proven to outperform even the best algorithms designed to

solve the TSP problem (Johnson & McGeoch, 1995). The search can also take

the name n-opt if more than two paths are considered at once.

The general solving process of a local search algorithm involves improving an

existing solution step by step by exploring neighboring solutions and adopt

those that appear promising, with the hopes of ending up at the goal. As a

metaphor, imagine that you are in the middle of a maze and must find a way

out. You see four different paths available, and while none of the paths

immediately leads to the goal, the visible length of path A is longer than any of

the other paths. Judging by this information, path A seems the most promising

since it allows for the most movement. The view of a local search algorithm

very much resembles this example. It explores neighboring solutions and

applies changes on a local level to traverse the search space. Just as a human

exploring a maze, its terminating conditions are either reaching the goal

3 - Related methods

15

(getting out of the maze) or exceeding the time limit (giving up) stating how

close to the goal it came.

Local search algorithms face an important challenge with local optima. To

understand this dilemma, consider a simple search algorithm known as hill

climbing. In this algorithm, a blind, one-dimensional man traverses a x*sin(x)

graph starting from x=0 with the hopes of reaching the highest peak, starting

at x=0. He will reach the first peak at x=2 thinking he has reached the highest

point, while in fact there are other mountains in the distance which would in

fact be higher peaks than where he is currently at. In more mathematical

terms, the peak found by the blind man is a local maximum but not the global

maximum he was seeking. This problem of getting stuck a local maximum is

one a local search algorithm is often faced with and there are various

improvements that can be conducted to avoid this issue.

To address the problem of local optima as well as speeding up the search, the

paper “A local search template” (Vaessens, Aarts, & Lenstra, 1997) starts from

the basic local search version and tries to come up with improvements. The

basic local search template considered by the author is the deterministic

iterative improvement algorithm which is another name for the previously

described example of local search when finding the way out of a maze. As a

recap, consider a local search algorithm that explores all its immediate

neighbors before picking the most promising one. The algorithm then accepts

this best neighbor as its current states and continues the search from there

until no further improvements can be made. The authors suggest various

categories of improvements to this algorithm:

• Exploring many neighbors at once, resulting in a multi-neighbor

search. This is the opposite of the single-neighbor search which is

referred to as the best improvement algorithm.

• Starting the search from various starting points, referred to as a multi-

start search.

• Allowing the neighbor function to accept worse states on certain

occasions to avoid getting stuck in local optima, called heat based

algorithms.

• Replacing the concept of current solution with a population. This

approach takes the name of a genetic algorithm.

3 - Related methods

16

• Alternating between different neighborhood functions to try to get the

best of multiple worlds. This is referred to as alternating

neighborhood search.

The remainder of this section covers each of these proposed improvements in

turn and analyze how they can be used to model the local search for the

formula analysis. Throughout these subsections, it is important to keep in

mind that algorithm running time is not an issue, since this time is negligible

compared to the time it takes to perform a test (at least an hour). Instead, a

desirable property of the formula analysis is to perform the local search while

doing as few “reads” as possible. The reason being that a test of three hours

needs to run for every such read and thus the time available for the algorithm

decreases very quickly as multiple reads are being conducted.

3.2.1 Heat based search

This category considers two algorithms, simulated annealing and tabu search.

Simulated annealing is an algorithm that involves using a random variable

which allows the local search to sometimes accept moving to a worse

neighbor compared to the current state. The goal of this is strategy is for it to

avoid getting stuck in local maxima and continue the search for the global

maximum, initially by moving to a worse solution. Its name comes from the

annealing process in the metal industry, which is used when shaping metal

objects. Initially, the material must be very hot for the worker to be able to

shape it and as the material cools, it becomes increasingly robust. The same

strategy is used in the simulated annealing algorithms; initially the

temperature is very high, allowing the algorithms to examine far worse

neighbors but as time progresses, the temperature decreases and the

algorithm has less margin in choosing worse solutions. Finally, after a fixed

amount of runs the temperature is so low that the algorithm is only allowed to

improve by moving to neighboring solutions by which time it has hopefully

found the global optimal value (Jacobson, 2013), (Carr, 2017).

Tabu search has a similar approach to simulated annealing but was originally

intended only as a higher-level heuristic upon which one can implement

another local search algorithm that runs with the help of the tabu search. Just

as simulated annealing, its goal is to allow that local search to surpass local

optima by accepting worse solutions. The difference however, lies in that tabu

3 - Related methods

17

search utilizes a list of previously visited states, tabu lists, which it refers back

to whenever the local search finds a spot where there are no improving

neighbors available. The tabu lists can hence be perceived as an aid for a local

search algorithm to be used whenever there are no improving moves. This

would allow any local search algorithm to be combined with tabu search to

reach the global optimum while avoiding local maxima traps (Gendreau,

2002).

To understand whether this category of solutions improves the local search

for the load reduction discovery, it is important to realize in what scenarios

local maximum are a problem and when they are not. For a monotonic

function, there are no local maxima and thus any maximum value found is also

the global maxima. This might seem obvious, since the maximum value y of a

monotonic increasing function would be the greatest assignment of x.

However, consider the case where the objective is to find the minimum

assignment of x in a monotonic increasing function such that f(x) > c where c

is a constraint given by the problem. Here, the optimal is not the greatest

value of x but the lowest values of x that fulfills the constraints.

As it happens, the function that maps traffic load to formula measurements is

of the monotonic increasing type where the goal is to reduce traffic as little as

possible while still reaching zero on the formula measurements. A discretized

version of this function could then be represented by a sorted array for which

binary search is an efficient solver, with the expectation to conduct log(n)

reads before the optimal solution is reached. For these reasons, the search

algorithm does not need help finding optimal solutions since it does this

already, instead a strategy is sought that could potentially reduce the running

time. Following this line of reasoning, neither simulated annealing nor tabu

search improves performance since both algorithms involves more reads

without improving the solution. The reason being that local optima are not a

problem due to the monotonic nature of the explored function.

3.2.2 Multi-start search

Another interesting improvement option is the multi-start search. There are

at least two different arguments for utilizing this strategy; one is to avoid local

optima and the other is to improve the running time where parallelism is

available. To understand this, consider how multiple starting points decided

3 - Related methods

18

with randomness allows the search to hill climb from various valleys with the

hopes of reaching various mountain tops. This information can then be used

to find the best mountain top which also hopefully is the global optimum. As it

happens multi-search has proven to be more efficient than algorithms such as

simulated annealing in the TSP problem (Boese, Kahng, & Muddu, 1993),

(Johnson, Local Optimization and the Traveling Salesman Problem, 1990).

The next question is to decide how many different starting points to use. The

first intuition is to use as many starting points as the amount of parallelism

available based on the reason that each starting point can then run on its own

thread and thus the exploration of each starting point does not interfere with

the available resources of exploration paths that belong to other origins.

Nevertheless, the multi-start strategy could also be employed in environments

which are single-threaded with the idea of immediately disposing suboptimal

starting points. As an example, consider the optimal load reduction is 17 while

the starting points chosen are 10, 20 and 30. First a test of reduction 10 is

executed, where there would still be indications of bad cells since the

reduction is not low enough. Next, the test of reduction 20 is run, where the

formula shows 0. From this information, it can be deduced that the optimal

load reduction lies between 10 and 20 and thus there is no reason to run

further starting points.

It is also worth mentioning that due to the nature of the formula analysis, the

multi-start search is not applicable. The reason being that the original test

serves as the starting point for the first iteration, hence the algorithm cannot

introduce new starting points. Instead, pseudo-starting points could be

introduced after the first test, i.e. in the second iteration. This would allow the

algorithms to further improve the solution by finding neighbors to these

second iteration starting points.

Nevertheless, it is not clear that this strategy is an improvement to binary

search. Without problem specific information available to the algorithm,

binary search runs quicker due to the monotonic increasing property of the

load/formulae function. Inclusion of problem specific information in the form

of rules is covered in the hybrid algorithms section.

3 - Related methods

19

3.2.3 Genetic algorithm

The most ambitious of all the improvements categories would be to try to

optimize the solver by employing a genetic algorithm. This kind of algorithm,

like its name implies, relies on survival of the fittest when it comes to

exploring neighboring solutions. In a living subject, this process is reflected by

chromosomes where those with the highest fitness tends to be the ones to

survive and reproduce new offspring. When the amount of offspring is zero,

this algorithm does not do a whole lot because it would then progress as

quickly as any other. The advantage comes when considering parallelism. If

the genetic algorithm can consider multiple different solutions (or offspring)

simultaneously, it can use the results to analyze which of these solutions that

are better fit to the problem configuration (Smita Sharma, 2013).

However, for a genetic algorithm to be complete, it must also include the

concepts of mutation and crossover – both of which have biological

counterparts. Mutation symbolizes the introduction of randomness into the

algorithm by allowing changes to occur with low probability. The purpose of

mutation in the genetic algorithm is to make sure it has the capacity to explore

the entire search space, again related to not being stuck in local minima or

maxima. For the same reason, the mutation cannot occur with high probability

because this would transform the algorithm into a random search which is not

very effective. Crossover on the other hand, details how an offspring is a

combination of the properties of its parents. However, in the special case

where there is only one parent, crossover cannot really be accomplished since

its very nature implies there are genetic information in two parents which is

combined.

This kind of implementation of a genetic algorithm for solving a hill climbing

problem has also been discussed by (Smita Sharma, 2013). They accomplish

this by replacing the chromosome evaluation part of a regular genetic

algorithm with a hill climbing search. This form the general algorithm

structure which is defined as:

1. Initialization: The algorithm starts with an initial set of solutions which

are used as the basis for the future search.

2. Selection: The most fit are chosen to continue the search from based on

some sort of evaluation function.

3 - Related methods

20

3. Hill climbing: A hill climbing algorithm is used to produce offspring of

each solution, i.e. moving each solution to a neighbor.

4. Recombination: This is where mutation and crossover occur.

5. Replacement: The original selection of solutions is replaced with the

result from this iteration and a new selection phase is entered.

This process can then be iterated until the result is good enough, based on

some sort of measurement or time limit. The authors then proceeded to

compare a regular genetic algorithm with the hybrid algorithm, utilizing the

hill climbing search on certain instances of the TSP problem. The results

showed that that the hybrid solution found shorter tours in all cases which

implies that the hill climbing addition can be a useful tool in optimizing hard

problems. Again, it is worth stressing that for this solution with genetic

algorithms to be useful in the problem of this thesis, there has to be multiple

tests executed simultaneously, such that many offspring solutions can be

investigated in each iteration. Due to the current high demand of test

execution time, this is unlikely to happen and thus a genetic algorithm is

abandoned for this thesis.

3.2.4 Multi-neighbor search

Much alike the multi-start strategy of local search, the multi-neighbor

approach involves branching to run not only the best immediate neighbor, but

to run the algorithms with K different neighbors after which the search

continues from these. This approach, which is also referred to as K-nearest

neighbor, carries the same advantages as multi-start search but is more

ambitious since it branches the solutions at every intersection rather than

only branching at the starting point. Therefore, it reduces the chances of being

stuck in local optima while also making effective use of available parallelism.

However, since it has previously been concluded that local optima are not an

issue for the formula analysis, the issue when considering this approach is

whether it can speed up the search or not assuming no parallelism is

available. The answer to this question is very much alike that to multi start,

with no problem specific information available this does not present an

improvement to a simple binary search. It is however possible that such

problem specific information could make a case for usage of multi-neighbor.

Including problem specific information is discussed further in the hybrid

algorithms section.

3 - Related methods

21

3.2.5 Alternating neighborhood search

Another attempt to improve the result and speed of a local search algorithms

it to use alternating (or variable) neighborhood search. The goal of this

approach is to, similar to heat based search strategies, allow the local search

to sometimes jump to solutions that would normally not be considered by the

algorithm. This is done through a process known as shaking, where a point

from the K:th neighborhood at a certain stage is used to move the search into

fresh territory (Hansen & Mladenovic, 1999). However, as the main idea of

this improvement is to avoid local minima it is not very promising from the

load reduction discovery for already mentioned reasons.

3.3 Knowledge based systems Another approach to solve the formula analysis is to use inference on a set of

problem specific information to deduce the traffic removal for each route. This

approach differs from search algorithms in this aspect, since the search

strategy is completely autonomous while knowledge based systems contain a

database of information the algorithm can use to infer and deduce solutions.

The advantage of this kind of algorithm is that the logic behind the solutions

proposed by the algorithm can be presented so that a human can follow the

deduction process. (Liao, 2005).

While there are a multitude of different approaches to this kind of algorithm,

most share some common traits. The systems need to store information

provided by humans in what is known as a knowledge base. Built upon this

knowledge base is an inference engine which contains the algorithm for

deducing solutions based on the problem provided and the rules contained in

the knowledge base. The inference engine can operate in one of two ways;

either by forward chaining which starts the problem and then applies rules to

try to reach the goal or by backwards chaining which starts with the goal and then tries to apply rules which leads to the problem.

These steps can be executed in iterative steps to apply rule after rule to from a

deduction chain from the problem to the solution. Shu-Hsien Liao (Liao, 2005)

lists a multitude of different variation of logic based systems. Those of interest

to the load reduction problem are:

• Rule based system

3 - Related methods

22

• Fuzzy expert systems

• Training based systems

The remainder of this section covers each approach and how they can be used to solve the formula analysis.

3.3.1 Rule based

The defining part of a rule based system is that its knowledge base consists of

rules, commonly if-then statements. The inference engine of such an algorithm

operates by trying to match rules based on the case stated in the rules which

is a three-step process: finding candidate rules to be used on the current state,

chose which rule to adopt should there be more than one and finally apply the

rule to reach a new state. This is an iterative process that advances the state of

the initial solution closer to the goal with each step (Liao, 2005).

3.3.2 Fuzzy logic

The use of fuzzy logic is best seen as an improvement that can be applied to a

rule based system. The idea is to allow the algorithm to deal with

uncertainties rather than absolute true and false values with the goal of easier

being able to represent human reasoning (Liao, 2005). This is done by letting

variables assume any value between 0 and 1 rather than being restricted to

the endpoints. A way of looking at this strategy is that each such variable

represents a measurement regarding how truthful a variable is, rather than

seeing it as either true or false.

It is common for a fuzzy logic extension to be applied to a rule based system

(Bernard, 1987) (Chang & Liu, 2008). The point of this strategy is that the

rules can now be stated using fuzzy logic rather than Boolean terms, which is

often how humans perceive rules. Consider this example of a rule: “If the

water level is high then decrease the water flow slightly”. This statement is

easy for a human to understand and can also be stated with fuzzy logic.

However, with Boolean logic this rule does not make sense, due to phrasings

such as “high” and “slightly” which cannot be directly mapped to true or false.

For these systems to work, reality needs to be converted to fuzzy logic during

problem input and the back again during solution output. To accomplish these

tasks, a common strategy is to apply a so called fuzzifier (or more correctly a

fuzzification interface) and a defuzzifier (defuzzification interface) for the

3 - Related methods

23

system to interpret the input and to output the result of its operations. These

modules operate by using membership functions which fuzzify crisp data into

fuzzy sets. The purpose of these functions is to quantify how much the crisp

data belongs to the fuzzy set represented by that membership function. The

degree of membership is measured using a value between 0 (does not belong

to this group) to 1 (belongs to this group). An example of such a fuzzy set

could be: “To what degree can the water level L be considered high”.

Normally, the membership functions form a triangular shape when drawn in a

graph, with a peak in appurtenance in the middle and then decreasing in both

directions. Therefore, the membership function to the given fuzzy set could be

defined as the following (Lee, 1990):

• 0-10 ml: 0 (too low)

• 10-15 ml: 0.5 (could be considered high – lower bound)

• 15-20 ml: 1 (definitely high)

• 20-25 ml: 0.5 (could be considered high – upper bound)

• 25+ ml: 0 (too high)

This membership function can now be used to indicate whether the

previously described rule regarding water levels should be used or not, given

a certain water state. Note that the membership of high does not extend to

infinity, since water levels higher than a certain point could belong to another

fuzzy set, such as “critically high”. A similar membership function would of

course also need to be included to formulate a fuzzy set of what “slightly

decrease” in this rule means when defuzzifying back to crisp data in the

output of the system. When using the defuzzifier to map the fuzzy values back

to real world values the peak value is chosen. If the water level membership

function would be used to defuzzify, the peak value 17.5 would be chosen.

The notion of completeness is important when designing a rule based system

using fuzzy logic. This term implies that for any state there should be a legal

rule which applies, assuming the goal has not already been reached. To

accomplish this requirement, there is normally a rule in the knowledge base

which states that should no other rule’s condition be matched well enough

with the fuzzy sets then the output is no action (Lee, 1990). The purpose of

this rule is to make sure the system is not forced to apply a rule which would

move the state to a worse condition.

3 - Related methods

24

The advantages of using fuzzy logic to improve a rule based system are clear.

Fuzzy rules are easier to formulate for experts as the number conversion is

done separately. Also, fuzzy logic helps deciding which rules to use should

there be many that match the criteria since each rule has a measured degree

of matching based on the membership functions output. This allows the

engine to pick the rule that has the highest degree of matching and then

implement this one. The downsides however lie in that there is a loss of

precision. The accuracy can however be controlled by using smaller intervals

in the membership functions, resulting in more precisely defined fuzzy sets.

Despite this, fuzzy logic is too imprecise to be used in high accuracy systems.

3.3.3 Training based

A different approach to model a logic based system is to use machine learning

approach. These strategies are based upon the algorithm learning

relationships from previous examples rather than using human supplies rules

as was the case with rule based systems. Two such examples are artificial

neural networks and case based reasoning which both are based on receiving

training data in terms of past experiences with known output. This

information is then used to configure the algorithms which can then be used

to provide answers to similar problems.

3.3.4 Analysis

The problems with using training based algorithms for the load reduction

discovery problem is that there is no real training data available. The

information available when configuring the algorithm is the advice of experts

which work better with rule based systems since these are designed to accept

data in this format. This is especially true for systems which employ fuzzy

logic due to their ability to accept rules in the human tongue.

3.4 Hybrid algorithms The conclusions thus far are that while local search algorithms are often

efficient, they struggle to outperform a simple binary search for the load

reduction problem due to the monotonic nature of the function. Logic based

systems on the other hand, especially rule based using fuzzy logic, seems

promising. However, this approach currently lacks the ability to draw

conclusions based on its previous runs. This section discusses two search

3 - Related methods

25

improvements to the logic based system so that it uses information from

multiple runs more efficiently.

3.4.1 Interpolation with expert system

The easiest strategy to accomplish this goal is to accompany the logic based

system with an interpolation engine. This would allow the algorithm to make

deductions not only by the rules in its knowledge base but also based on the

tests that have already been completed by interpolating the test results. The

base assumption, also based on the expert’s knowledge, would be that there is

a linear relation between load and formula measurements. From this

assumption, each test completed would provide another piece of information

to an overdetermined linear system which could be solved by the least square

method to draw conclusions as to where the next point should be. This

deduction could be combined with the rule applied from the inference engine

to make a hypothesis where the solution is.

3.4.2 Optimizing binary search with expert system

Another strategy to improve the algorithm is to use the binary search strategy

previously explained in conjunction with the logic based system. The idea

here is to use the logic based system to advice the binary search where to

search next. On first thought, this might seem strange since the binary search

always looks at the middle element and then decides whether to continue

searching in the lower half or the upper half. However, by definition the

objective of the binary search is not necessarily to choose the middle element

but to choose the element based on minimizing statistical expectation such

that the solution is just as likely to be in the lower half as in the upper half. In

some cases, this pivot element is not the one in the middle, and this is also the

case with the load reduction problem. By using a logic based system, the

inference engine can provide deductions as to where to proceed with the binary search.

3.5 Flow networks Flow networks, originally suggested in (Ford & Fulkerson, 1962) is a graph

model used to visualize the problem of finding a maximum flow between two

points. While such problems can also be solved as a linear program, modeling

them as a flow network is simpler and makes the problem less

computationally expensive to solve. Despite the age of the original paper, the

3 - Related methods

26

problem of maximum flow is still discussed today and improvements are

discussed even in papers in 2018, such as (Ghaffari, Karrenbauer, Kuhn,

Lenzen, & Patt-Shamir, 2018) which discusses how distributed system can be used to solve maximum flow problems more efficiently.

The graph definition for a flow network is defined as (G, c, s, t) where:

• G is the underlying directed graph

• c is the capacity function of the arcs

• S is the source vertex

• t is the sink vertex

The solution to the maximum flow problem on a flow network is to find the

maximum flow from the source vertex S to the sink vertex T using the arcs of

the graph. Flow cannot be added or removed on any other vertex. In addition,

the capacity on each arc specify the maximum amount of flow than can pass

through that arc.

An example of an algorithm used to solve the maximum flow problem of flow

network is the Edmonds-Karp algorithm which has a maximum running time

of O(VE2) where V is the numbers of vertices and E is the number of arcs in

the graph G (Lammich & Reza Sefidgar, 2018).

4 - Method

27

4 Method This section covers the method used. First, the information presented in the

Background section is used to formulate the optimization problem. Then, the

optimization problem is modularized into smaller steps. Finally, the

implementation of each of these steps is covered.

4.1 Problem formalization With the test network covered, the problem can now be formalized using the

parameters available. The goal is to find traffic allocations for all routes such

that the node and cell level formulae are satisfied while maximizing the traffic

on each node. However, since the experts already have good knowledge about

maximum load the solver is not allowed to increase the amount of traffic on

any node from the testers’ initial configuration. The formalized problem

formulation is presented in equation 4-1 where fN and fC are the node level-

and cell level formulae while n, c, r and t represent node, cell, route and traffic.

Recall from the Terminology section that a route r is considered to belong to a

node n if there is a cell c that is a subcomponent of n and has an endpoint in r.

The term ∆tr reflects the change in route load prior to and after the

optimization, i.e. tr0 + ∆tr = tr. Finally, g(f) is the function returning a Boolean

value of whether the formula filter of f has passed.

argmax𝑡𝑟

∑𝑡𝑟𝑟

𝑠𝑢𝑐ℎ 𝑡ℎ𝑎𝑡

∑ ∆𝑡𝑟𝑟:𝑟∈𝑛

≤ 0

∀n, ∀fn: 𝑔(𝑓𝑛)

∀c, ∀fc: 𝑔(𝑓𝑐)

(4-1)

The optimization problem formulation of equation 4-1 can now be

modularized into steps. Separating the two formulae constraints into different

goals, yields the following objectives:

1. Find the maximum traffic assignment that satisfies the node formulae

requirement: ∀n, ∀fn: 𝑔(𝑓𝑛)

2. Find a traffic redistribution that satisfies the cell level formulae requirement: ∀c, ∀fc: 𝑔(𝑓𝑐)

Since cells are subcomponents of nodes, the solution to 1) form a constraint

when solving 2). That is, a traffic allocation that maximizes load while all

4 - Method

28

nodes pass the set-out formulae constraints implies that all nodes have

achieved the tolerable amount of traffic. Thus, solving 2) is a matter of

redistributing load in each node such that each cell also passes the formulae.

Given this problem breakdown, the node traffic discovery step of solving the

problem is defined as the solution to 1) and is formalized in equation 4-2.

Also, the call traffic optimization is defined as the solution to 2) given the

constraint of the node maximization, formalized in equation 4-3. Since both

models deal with traffic changes, the term ∆tr has been split into ∆tr’ and ∆tr’’, where ∆tr = ∆tr’ + ∆tr’’.

max∑𝑡𝑟𝑟

𝑠𝑢𝑐ℎ 𝑡ℎ𝑎𝑡

∑ ∆𝑡𝑟′

𝑟:𝑟∈𝑛

≤ 0

∀n, ∀fn: 𝑔(𝑓𝑛)

(4-2)

𝑓𝑖𝑛𝑑 𝑡𝑟 𝑠𝑢𝑐ℎ 𝑡ℎ𝑎𝑡

∀c, ∀fc: 𝑔(𝑓𝑐)

∑ ∆𝑡𝑟′′

𝑟:𝑟∈𝑛

= 0(4-3)

4.2 Node traffic discovery The general goal of the node traffic discovery is to find a traffic model that is

as harsh as possible while still being solvable. In this context, this translates to

minimizing the load reduction of each node without any of them showing

symptoms of instability. To achieve this goal, there are two main problems

that needs to be solved. The first problem is to find how much traffic that

should be removed from each bad node. This problem is henceforth referred

to as the node-level formula analysis. Additionally, there are in most cases

multiple different routes originating from nodes that can be targets for traffic

removal. Thus, the second problem, the node-level traffic redistribution

problem involves finding what routes any traffic reduction should be

conducted on in order to reduce the node load.

4.3 Cell traffic optimization The problem of solving equation 4-3 is referred to as the cell traffic

optimization. The main difference between the node traffic discovery and this

step is that while the former reduces traffic the latter must find a legal

destination to move that traffic to without removing any traffic. Thus,

4 - Method

29

passing all formula filters indicating bad cells is a matter of distributing some

load of those cells between the connected routes in such a way that each node

shows zero change in load. Complicating matters further is that traffic cannot

directly be moved between cells but are instead moved between routes. Thus,

two cells are affected by each move operation, since every route is connected

between two cells (recall figure 2-3).

In addition, traffic reallocation is also constrained by the handover types of

the different routes. These constraints have been set up by the stability team

since they favor testing the nodes’ capacity to handle handovers of certain

types over others. The internal priority order is depicted in equation 4-4 and

states that X2 is the most important handover type to fulfill followed by S1

and Irat and lastly intra. The purpose of this rule is to restrict the optimization

from removing traffic of a higher priority handover type in favor of one of

lower priority. As an example, removing traffic on X2 and adding traffic on S1

is illegal while the other way around is ok.

𝑋2 > 𝑆1, 𝐼𝑟𝑎𝑡 > 𝐼𝑛𝑡𝑟𝑎 (4-4)

Thus, the cell traffic optimization is also divided into two subproblems. It

requires a cell-level formula analysis to find the amount of traffic that needs

to be reduced on each cell. Then, the cell-level traffic redistribution

problem finds paths where traffic can be redistributed to and should there be

multiple ways of doing so, also calculates what traffic should be redistributed

to each route.

4.4 Solution strategy Based on the discussion in the Related methods section, solving the

subproblems discussed can either be done using the experts leading to a

knowledge based system or using algorithms operating based on data alone

which falls under the local search algorithm category. Despite the various

discussed improvements that can be made to the local search algorithm it is

still difficult to find a version that outperforms an interpolation algorithm, the

main reason behind this being the monotonic nature of the function that is

optimized. Heat based search provides no benefits while complexity of the

genetic algorithm approach means there is a lot of overhead due to the

mutation and crossover operation that needs to be carried out. Since the

4 - Method

30

number of test runs are limited this process is too costly. Nevertheless, the

multi-start and multi-neighbor solutions provided some potential

improvements should there be problem specific information or parallelism available.

Optimizing the tests using either expert systems or local search algorithms

can run multiple iterations while attempting to converge to the optimal

solution. The following sections covers the implementation of the previously

mentioned subproblems of an arbitrary such iteration. The idea is that

illustrate that if any iteration, including the first, configures the test closer to

the optimal solution then multiple iterations would also converge to the optimal solution.

4.5 Node level formula analysis The node version of the formula analysis from the second iteration and

forward is most candidly solved with an interpolation algorithm since node

traffic load has an approximately linear relationship with the corresponding

formulae measurements. Thus, the node-level formula measurements

relationship to the traffic load for a given node can be modeled using the

standard linear equation and solved with a minimum of two data points. For

the first iteration however, only one data point is available meaning another

solution is required.

Solving the case of one data point with a purely data-driven approach can be

accomplished using a separate experiment of multiple tests to estimate the

initial reduction. This experiment could be done by trying different traffic

reduction amounts in steps of five percent and observe whether the test then

passes the node formula measurements. There are however a couple of issues

with this approach. Primarily, the formulae of interest as well as the tolerance

levels vary from test to test meaning it is by no means guaranteed that the

reduction constant will remain stable between test cases or even between nodes.

Another option is to base the initial guess using expert formulated rules

meaning the hybrid algorithm discussed in 3.4.1 Interpolation with expert

system on page 25 is used. Thus, the expert part of the hybrid algorithm is a

rule of how generously the formula analysis should remove traffic from the

4 - Method

31

node in the first iteration. This rule can vary from test to test and can be

formulated either with Boolean or fuzzy logic. Also, the assumption of a linear

relationship between traffic and formula measurements is based on experts’

input and is thus also a part of the rule base for this hybrid solution. This

approach will be used for the experiment, further details on the rules used are

presented in the Experiment section.

4.6 Cell level formula analysis Like the formula analysis for nodes, the cell level formula analysis can also be

conducted using either a data-driven approach or an expert system. The

situation for cells does however differ from nodes in that cell formula

measurements relationship with load is non-linear (recall Figure 2-5) and

thus more difficult to model. Doing so in a purely data-driven approach is

however possible using a binary search strategy in the case of multiple

iterations. Nevertheless, it is more promising to also include a rule-based

system for the experts to provide a clue as to what pivot element to use.

Thus, a hybrid algorithm is proposed for the cell level formula analysis. The

algorithm discussed in 3.4.2 Optimizing binary search with expert system on

page 25 can be used to supplement the expert’s knowledge with a search

algorithm. This implies that for each iteration the algorithm considers the

previously highest traffic iteration that passed the formulae filters together

with the previously lowest traffic iteration that did not pass them. A rule

provided by the experts is then used to decide the pivot element in the available range.

4.7 Node level traffic redistribution For the node level traffic redistribution, the implementation method does not

have much impact. The reason is that any traffic this module redistributes

between cells is likely to be overwritten by the optimization phase for the cell

level traffic redistribution in the next iteration anyway. For completeness however, some different options are presented.

The data-driven solution is to simply distribute the traffic reduction evenly

between all routes connected to the node. This approach assumes the traffic is

close to correctly configured from the start and makes no effort to rebalance

the traffic between the cells without further information. Despite being the

4 - Method

32

simplest approach, this is by no means a bad idea for this reason alone; the

experts probably configured the traffic well from the start.

A rule-based option of redistributing load would be to try to put the traffic

reduction on routes connected to bad cells based on weights provided by

experts. This sounds promising but it is important to remember that basing

node level traffic on cell level formulae filters is illogical since modified node

traffic will affect the optimization in the next stage. Thus, the data-driven

approach is selected for the node level traffic redistribution.

4.8 Cell level traffic redistribution Since the cell-level traffic redistribution is part of phase two of the

optimization process it is restricted by the results of the node level traffic

discovery. Thus, any traffic move operation suggested by the cell-level traffic

redistribution must not violate this restriction, depicted in equation 4-5. Here,

∆tR‘’ is the delta traffic assigned during the optimization on each route.

∑ ∆tr′′ = 0

𝑟:𝑟∈𝑛

(4-5)

Assigning traffic allocations to routes without violating equation 4-5 is non-

trivial. It implies that for any bad cell that requires a traffic delta of -t there

must be cells on the same node that combined receives a traffic delta of +t for

the node level traffic load not to be altered. One way of ensuring these

conditions is to model the cell-level traffic redistribution as a max-flow

problem.

4.8.1 The flow network approach

When consider the cell-level traffic redistribution as a max flow problem, a

flow network is used to represent the architecture. Note that normally the

vertices in a flow network is by convention referred to as nodes, but due to

the confusion with architecture nodes (ENBs) in this context, the flow

network nodes will henceforth be referred to as vertices. Recall from section

3.5 that a flow network is described as (G, c, s, t) where:

• G is the underlying directed graph

• c is the capacity function of the arcs

• S is the source vertex

4 - Method

33

• t is the sink vertex

The underlying directed graph (G) is defined as follows. Each cell in the

architecture is represented by two vertices while the arcs (edges) represent

the set of non-intra routes in the architecture. One of the two vertices for each

cell is referred to as the plus vertex and implies that all the incoming flow to

this vertex is an addition of traffic on the corresponding route. The opposite is

true for the minus vertex. Any arc between two vertices of the same parity is

removed from the graph. This guarantees that flow in the graph will alternate

between plus and minus vertices. Thus, the node level traffic load will not be

altered since a solution to the max flow problem guarantees that there is no

loss or addition of flow at any vertex. Additionally, arcs are added to form a

complete graph between all plus vertices of the same node. The same is also

done for all the minus vertices of each node. These arcs represent the

property that flow can be freely distributed in a node, as long as all incoming

flow is also going out without changing parity inside the node. Under these

conditions, the node traffic restriction is upheld.

Now assume that there is a bad cell in one of the vertices. All arcs connecting

this vertex to other vertices of the same node are removed from the graph as

they cannot be used to reduce the load of the bad cell. The source vertex (s)

is now added and connected to the plus vertex of the bad cell. The other plus

vertices of the same node will be connected to the sink vertex (t). This

resulting graph is visualized in Figure 4-1. All non-directed arcs represent arcs

in both directions. A path of arcs connecting the source to the sink has been

highlighted, indicating that this path can be used to reduce load of cell one in

node three.

4 - Method

34

Figure 4-1: The flow network graph

To correctly solve the max flow problem, there must be a capacity on each arc.

This represents the maximum flow that a specific arc can support. The

capacity function (c) for this flow network is defined as follows:

• All arcs directing flow from a plus to a minus vertex have a capacity

according to the current traffic load of the route they represent. The

reason is that no more than the current traffic load can be removed

from any route.

• The arc between the source and the bad cell have a capacity according

to the formula analysis of mentioned bad cell. This is the amount of flow

the algorithm should try to accommodate in the network to solve the

bad cell.

• Any other arc has infinite capacity.

Up until this point, routes between cells of the same node has not been

considered. However, such intra-routes can also be used to solve the cell-

level traffic redistribution. In fact, they can be added to the flow similarly to

the inter-routes. Consider Figure 4-2, where there is only one node with four

cells, with one bad cell in C1. All arcs have not been drawn out due to

4 - Method

35

cluttering, but the same rules as the previous definition of the flow network

applies. That is, all good cells are connected to the sink vertex and there is a

complete graph of all vertices of same parity (except the bad cell). In addition,

the present intra routes have been drawn out between the vertices

corresponding to its route, but only from one parity to another. Just as before,

the capacities of intra routes from a plus to a minus vertex correspond to the

current load. A valid path of arcs has been highlighted from the source to the

sink vertex.

Figure 4-2: The flow network with intra routes

A max flow solution to this network would also be a solution to the bad cell

when interpreting flow as traffic delta. An algorithm such as Ford-Fulkerson

(Ford & Fulkerson, 1962) can find a solution to the max-flow problem in

polynomial time with respect to the size of the graph. The number of vertices

does increase as each cell is represented by two vertices. Nevertheless, this

increase is linear and thus the complexity of solving the cell-level traffic redistribution using this flow network is still polynomial.

4.8.2 Multiple bad cells

While the flow network solution has been shown to optimize one bad cell, the

question is now how this approach deals with multiple bad cells. Connecting

4 - Method

36

the source node to all bad cells and similarly connecting all good cells of the

same node to the sink yields one graph. However, this allows for flow from the

source node of one bad cell to a good cell of another node, thus violating the

node traffic restriction of equation 4-5. Another problem that needs to be

solved is how to deal with the handover type priority of equation 4-4 which

states that handover types have an internal priority dictating what handover

traffic that can be added in favor of other such handover types. These two

issues will now be covered.

4.8.3 Handover types priority

The issue regarding handover type priority can be resolved during the path

exploration part (e.g. breath-first search) of the Ford Fulkerson. By keeping

track of the highest priority arc to a minus vertex explored thus far for each

path, the exploration of that path may not explore arcs of a lower priority to a

plus vertex. This ensures all paths that are then use to assign flow are valid

with regards to handover types.

4.8.4 Net flow between nodes

Similar to the handover-type priority, the problem of assigning flow form one

bad cell to cells of another node can be resolved during the path discovery. By

keeping track of the bad cell used in the start of path, the same path may only

use the sink node of that cell. This ensures that the node traffic restriction of

equation 4-5 is not violated during the max flow algorithm.

5 - Experiment

37

5 Experiment This section covers the investigation that was conducted using output from

the test networks of the stability team at Ericsson. First, the configuration of

the algorithm’s parameters is presented, followed by the architectures used

for the experiment. Finally, the results are listed.

5.1 Node level formula analysis Since the node-level formula analysis is based on expert rules for the first

iteration of test optimization, a rule must be defined for this purpose. A

discussion with the experts has yielded that the initial node traffic reduction

should be around 10% of the original traffic load. However, using fuzzy logic;

this estimate can be enhanced to work across different scenarios of traffic. To

begin with, if the initial traffic load of the node is less than twenty, the 10%

rule would mean that the reduction is less than two. The traffic simulated tool

used at Ericsson supports only changes at two or more which would make this

traffic reduction meaningless. Thus, this rule can be enhanced by using a fuzzy

logic rule phrased as “if the initial traffic amount is very low, then the traffic

reduction should be the minimum change supported by the tool”. The

membership function to “very low” in this case would be an initial traffic load

of less than 20 while the defuzzifier would state that the “minimum supported

change” would equal two. For this thesis, all other initial traffic amount maps to a reduction of 10%. This rule is depicted in equation 5-1.

ℎ0(𝑛) = 2 𝑖𝑓 𝑡𝑛 < 200.1𝑡𝑛 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

(5-1)

5.2 Cell level formula analysis As for initial reduction of cell load, the discussion with the experts also yielded

that this reduction should be around 10%. However, there are several lapses

with using only a simple 10% rule which is now highlighted. To begin with,

the purpose of the 10% rule is to reflect the fact that the experts are likely to

have made an initial good estimate of what the traffic should be on each

component. This is however much more likely to be true for nodes rather than

cell. The reason is that nodes have been established to follow a linear

relationship between load and formula. Cells on the other hand have been said

to follow an unpredictable relationship between the same units. Thus, it

5 - Experiment

38

makes sense to enhance the cell level initial guess with fuzzy logic to reflect

the current state of the experts’ guess as to what the reduction should be.

To begin with, consider the case where we have yet to detect a traffic load for

a specific cell that results in it being good (i.e. a lower bound). In such

situations, there is no knowledge as to what the next guess should be and thus

the best available information is to follow the expert’s advice of 10%

reduction of the observed upper bound. If on the other we have observed both

a lower bound and an upper bound, then this information can be used to

formulate a guess based on linear interpolation between the two values, as

described in the algorithm section. Choosing between following the experts

and using the linear interpolation value is now the challenge. A reasonable

way of making this choice is to observe the difference between the lower and

upper bounds. If the distance between these values is large, then it would

appear that the experts initial guess is bad since the span of possible solutions

is large. If the distance between the high and low values are small on the other

hand, the experts probably estimated the traffic well, and the 10% rule can be

used.

This yields the following fuzzifier: If there is no previous knowledge of good

cell values (membership: unobserved lower bound) then use the experts’ rule

or 10%. In addition, if there is an observed low value close to the observed

upper bound (membership: difference is below 100) then use the experts’ rule

of 10%. If none of these conditions apply, use binary interpolation. This is

illustrated in equation 5-2 where 𝑂(𝑐) is the upper bound and Ω(𝑐) is the

lower bound of a given cell.

ℎ(𝑐) =

0.1𝑡𝑐 𝑖𝑓 Ω(𝑐) = ∞

0.1𝑡𝑐 𝑖𝑓 𝑂(𝑐) − Ω(𝑐) < 100

𝑂(𝑐) − Ω(𝑐)

2 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

(5-2)

5.3 LTE test configuration The test used for the experiment had an interval sampling rate of 15 minutes

and the tests used as input for this experiment had ran for durations of 2-3

hours.

5 - Experiment

39

5.3.1 Formulae used

The formulae used for the experiment are:

Node level (tolerance: <=0%):

• Handover reject frequency due to overload

• Handover reject frequency due to high load

• Handover reject frequency due to high intensity

• Handover reject frequency due to software limitations

Cell level (tolerance: >=97%):

• Accessibility status

• Mobility status

• DTX rate

5.3.2 Test cases

As previously mentioned, the investigation is performed on three different

architectures: low, standard and many routes. The few routes configuration is

designed to test the algorithm’s capabilities to find traffic reallocations despite

having limited options. A description of the configuration is seen in figure 5-1.

The standard routes configuration is the architecture used for these specific

nodes and cells even before it was included as part of the experiments of this

thesis. Thus, it represents a regular test with a standard number of routes. Its

architecture is described in figure 5-2. The final architecture is given an

excessive number of routes to test the algorithm’s ability to make the most

use of the options available. Its architecture is depicted in figure 5-3.

5 - Experiment

40

Figure 5-1: The few routes architecture

Figure 5-2: The standard routes architecture

Figure 5-3: The many routes architecture

5 - Experiment

41

5.4 Evaluation metrics To understand the algorithm’s impact on the traffic situation, two statistics

have been chosen in accordance with the experts. These are:

Cell solve score (CSS), which reflects the system’s capabilities to reduce cell

traffic where required. It is calculated as the average cell solve quota across

the bad cells, where the cell solve quota is the sum of traffic removed from a

bad cell divided by the formula analysis estimation for the same cell. Thus, the

higher the score the more successful the algorithm has been at removing

traffic from the bad cells. The calculation is depicted in equation 5-3 where CB

denotes the set of bad cells.

𝐶𝑆𝑆 =

∑ ∑ min (∆tr′′

𝑔(𝑐), 1.0)𝑟:𝑟∈𝑐𝑐:𝑐∈𝐶𝐵

|𝐶𝐵|(5-3)

Cell squared error score (CSES) reflects the extent to which the algorithm’s

traffic modifications could cause more bad cells in future iterations. It is

calculated as the sum of the squared traffic increase across all cells

normalized by the number of bad cells. The squaring is used to indicate that

high traffic increases on cells are bad since those cells are more likely to

become bad in the next iteration. Thus, a lower CSES score is better. The

calculation is depicted in equation 5-4 where again CB denotes the set of bad

cells.

𝐶𝑆𝐸𝑆 =∑ ∆t(𝑐)2𝑐:∆t(𝑐)>0

|𝐶𝐵| 𝑤ℎ𝑒𝑟𝑒 ∆𝑡(𝑐) = ∑ ∆tr

′′

𝑟:𝑟∈𝑐

(5-4)

5.5 Results A database of tests results was prepared by the stability team for the

experiment. This database contained 10 tests whereof each was optimized

using each architecture, thus a total of 30 optimizations were performed. Out

of the 10 tests, there were five groups of two for each number of bad cells,

ranging from 1 to 5. A complete list is shown in Table 5-1. The motivation for

this selection is that the number of bad cells has a substantial effect on the

performance of the optimization as there are fewer good cells that can accept

5 - Experiment

42

more load. The test cases were selected randomly from a time period of one

week.

Bad cells Few routes Standard routes

Many routes

Sum

1 2 2 2 6 2 2 2 2 6 3 2 2 2 6 4 2 2 2 6 5 2 2 2 6 Sum 10 10 10 30

Table 5-1: Experiments conducted

For each combination of architecture and bad cells count the optimization was

performed using the method and configuration described. Afterwards, the

evaluations metric CSS and CSES were calculated. The result with the lowest

CSS score for each combination was selected for each such set of experiments

is presented in Table 5-2. The reason for the exclusion of the high CSS result

for each experiment was to reduce the effect of low-hanging fruit affecting the optimization positively with so few tests available.

Bad cells

Few routes Standard routes Many routes

CSS CSES CSS CSES CSS CSES 1 100 200 100 200 100 100 2 0 0 0 0 100 275 3 30 81 33 33 37 23 4 28 6 53 36 53 48 5 28 15 40 52 60 120

Table 5-2: Worst result for each subset of experiments

The evaluation metrics was then aggregated based on the architecture, thus

eliminating the number of bad cells as a variable. The reason is that the

number of bad cells is not an actual point of interest to the experiment, rather

a means of ensuring fairness. The median, mean and standard deviations per

architecture was calculated for each such aggregation. The results are

displayed in Table 5-3.

5 - Experiment

43

Statistic Few routes Standard routes Many routes

CSS CSES CSS CSES CSS CSES Median 30 15 40 36 60 100 Mean 37 60 45 64 70 113 Standard deviation

37 85 36 78 28 98

Table 5-3: Aggregated statistics for the full experiment

6 - Discussion

44

6 Discussion This section analyzes the results and list the issues, improvements and

alternative solutions.

6.1 Analysis Upon examining Table 5-3 there is a linearly increasing trend between the

architecture size and CSS score. In addition, the CSS variance decreases with

architecture size. This indicates that the algorithm makes consistently

effective use of the available traffic allocation options to optimize traffic and

avoid further bad cells.

However, another visible trend is that the CSES score also increases as the

architecture grows larger. One reason for this phenomenon is that all traffic

removed from bad cells must be put somewhere else, so if the algorithm

removes more traffic from the bad cells then this is also compensated for by

adding further traffic on some good cells, increasing the CSES score. An

especially good example is the test case with two bad cells, which was very

difficult to solve. Neither in low or standard routes were there a solution

available, while the solution found in the many routes architecture required a

relatively heavy investment in other good cells in order to solve the bad cells

entirely. As this was the by far highest CSES score in the entire results table, it

had a severe impact on the CSES mean score for the many routes architecture.

Other trends that can be observed in the results is that the CSS score

decreases with a high number of bad cells. This is to be expected since solving

all the bad cells when there are more such cells than good is a difficult

operation due to the limited options of reallocations. The reader might also

wonder why the few routes have such a high CSES score for one bad cell. This

is a phenomenon which can occur specifically when running with one bad cell

since the routes could potentially be very bad for this cell, especially if the

handover types of these few routes prevents traffic reallocations that would

yield a lower CSES score. As the number of bad cells increase however, this

phenomenon in not as influential on the total score as the algorithm is given

more options in terms of which routes to use.

6 - Discussion

45

6.2 Criticism To begin with, consider some of the issues by using the proposed solution in

this thesis. The implemented solution implies that by reallocating the mobile

traffic (UEs that move between cells) the tests can be reconfigured to reduce

the number of bad cells. However, not all actual users in the real world are

mobile. One example is an airport under harsh weather conditions, in which

case there would be multiple static UEs connected to the nodes, but not a lot

of mobile UEs.

To put this in a test environment, consider a cell with a max capacity of 1000

UEs which at the time of configuration is given 900 static UEs and 150 mobile

UEs. The algorithm reduces the amount of mobile UEs by half to reach the

configuration 900 static UEs and 75 mobile UEs, thus passing the capacity

requirements and the test has passed. However, the ratio between static and

mobile UEs has been severely impacted, from 14% mobile UEs to 7%. Thus,

the test has been passed during different circumstances than was initially

configured. Altering the rations of mobile UEs affect the type of traffic being

teste since certain traffic types more frequently occur for static UEs. One

example is video traffic is more likely to occur for static UEs than mobile UEs.

Thus, by changing the ratio between mobile and static UEs without changing

the traffic profiles, it is possible the algorithm could create unrealistic tests

which is a problem. To resolve this issue, the test configuration options

available to the solver must be expanded.

6.3 Alternative solutions An alternative approach to the proposed algorithm which has been briefly

discussed in the Related methods section is to use a learning system instead.

This would involve discovering configurations by using the results of multiple

tests as training data. While the traffic redistribution algorithms are

autonomous as they are, the formula analysis currently depends on the

expert’s rules to make decisions. Thus, this module could be replaced by a

learning algorithm. To make this work, the new algorithm would need to take

the input formula measurements into consideration when doing this

modelling since these parameters impact how large the required traffic

reduction is for the bad architecture components. Using a learning algorithm

could help avoiding undershooting the optimal load in cases where the logic

6 - Discussion

46

used by the experts would cause the nodes to run with less traffic that they

could potentially handle.

The problem with learning algorithms however is that they require training

data in potentially large amounts. As it is, there are no such records kept as

they are deleted after a certain amount of time. Thus, there would first need to

be a data collection phase, storing information about relationships between

traffic amounts and formula measurements. A sample of different learning

algorithms would then have to be tested on this data and then evaluated

based on precision and recall with the goal of coming up with a solution that

would outperform the rules provided by the experts.

A way of bypassing the lack of available data points could be to approximately

model the test network in a simulation and train a learning algorithm by

running it on the simulated network. Agents trained using different

architectures on the simulated network could then be evaluated against the

actual test networks. This would avoid the expensive process of producing

training data on actual test networks. It is however likely a challenge to

properly simulate the test networks and how they affect the formulae used in

the optimization process suggested in this thesis.

7 - Conclusion

47

7 Conclusion To conclude, the objective outlined will be covered in order.

The primary objective was to find an algorithm which reliably suggest

improvements to a test while accommodating all the experts’ requirements

and minimizing the reduction of handovers. The proposed solution used node

level load as a constraint in the optimization problem and forces a handover

amount that is as high as possible. The optimization step then enforces the

expert’s requirements regarding handover type priority during the solution

discovery.

The secondary objective was that the suggested algorithm should be able to

run multiple iterations and use results from more than one set of test results

during optimization. The method discussed has covered this case by

introducing hybrid algorithms which utilize both expert’s rules and search

based algorithms on data from multiple iterations. However, this was not

tested in the experiment and thus no conclusions can be drawn as to whether

this works in practice.

Finally, to answer the problem statement:

Can an algorithm that suggests optimizations to LTE mobility tests be

constructed? If so, could such algorithm be designed to converge using

test results of multiple iterations?

Yes, such an algorithm can and has been constructed which has proven to

work on three test cases. As the test cases grew larger the algorithm managed

to optimize better, suggesting it works in a wide variety of cases. No definitive

conclusions can be drawn as to how it works in multiple iterations however as

this part was not verified in the experiment due to restrictions in the

availability of the test platform.

8 - References

48

8 References Bernard, J. (1987). Use of a Rule-Based System for Process Control. Cambridge,

Massachusetts, USA: Massachusetts Institute of Technology.

Boese, K., Kahng, A., & Muddu, S. (1993). A new adaptive multi-start technique

for combinatorial global optimization. Los Angeles, CA. USA: UCLA

Computer Science Department. Retrieved from

http://www.sciencedirect.com/science/article/pii/016763779490065

5

Carr, R. (2017, August 4). Simulated Annealing. Retrieved August 8, 2017, from

Wolfram MathWorld:

http://mathworld.wolfram.com/SimulatedAnnealing.html

Chang, P.-C., & Liu, C.-H. (2008). A TSK type fuzzy rule based system for stock

price prediction. Expert Systems with Applications 34, 135-144.

Ford, L. J., & Fulkerson, D. (1962). Flows in Networks. Princeton, NJ: Princeton

University Press.

Gendreau, M. (2002, July). AN INTRODUCTION TO TABU SEARCH. Retrieved

from

http://home.ifi.uio.no/infheur/Bakgrunn/Intro_to_TS_Gendreau.htm

Ghaffari, M., Karrenbauer, A., Kuhn, F., Lenzen, C., & Patt-Shamir, B. (2018).

Near-Optimal Distributed Maximum Flow. Retrieved 05 26, 2019, from

https://arxiv.org/pdf/1508.04747.pdf

Hansen, P., & Mladenovic, N. (1999). Variable neighborhood search: Principles

and applications. European Journal of Operational Research, 449-467.

Jacobson, L. (2013, April 11). Simulated Annealing for beginners. Retrieved

from The Project Spot: http://www.theprojectspot.com/tutorial-

post/simulated-annealing-algorithm-for-beginners/6

Johnson, D. (1990). Local Optimization and the Traveling Salesman Problem.

Murray Hill, USA: AT&T Bell Laboratories. Retrieved from

http://dl.acm.org/citation.cfm?id=684359&CFID=937142151&CFTOKE

N=88888369

8 - References

49

Johnson, D., & McGeoch, L. (1995). The Traveling Salesman Problem: A Case

Study in Local Optimization. Retrieved from

https://www.cs.ubc.ca/~hutter/previous-earg/EmpAlgReadingGroup/TSP-JohMcg97.pdf

Lammich, P., & Reza Sefidgar, S. (2018). Formalizing the Edmonds-Karp

Algorithm. Retrieved 05 26, 2019, from https://www.isa-

afp.org/browser_info/current/AFP/EdmondsKarp_Maxflow/document.

pdf

Lee, C. (1990). Fuzzy Logic in Control Systems: Fuzzy Logic Controller - Part 1.

IEEE TRANSACTIONS ON SYSTTMS. MAN, ANID CYBERNETICS, 20, 404-

418. Retrieved from http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=52551

Liao, S.-H. (2005). Expert system methodologies and applications - a decade

review from 1995 to 2004. Expert Systems with Applications, 93-103.

Retrieved from

http://www.sciencedirect.com/science/article/pii/S09574174040009

34

Poole, I. (n.d.). GSM Network Architecture. Retrieved March 28, 2017, from

Radio-Electronics.com: http://www.radio-

electronics.com/info/cellulartelecomms/gsm_technical/gsm_architecture.php

Smita Sharma, G. G. (2013, December). M.Tech (CSE), Department of

Computer Science & Applications KU,. International Journal of Computer

Science and Mobile Computing, 27-34. Retrieved from Hybridizing

Genetic Algorithm with Hill Climbing in Replacement Operator:

https://www.ijarcsse.com/docs/papers/Volume_3/12_December2013/

V3I12-0158.pdf

Strang, G. (2016). Introduction to Linear Algebra, Fifth Edition. Massachusetts,

USA: Massachusetts Institute of Technology. Retrieved August 13, 2017,

from Massachusetts Institute of Technology:

http://math.mit.edu/~gs/linearalgebra/ila0403.pdf

8 - References

50

Vaessens, R. J., Aarts, E. H., & Lenstra, J. K. (1997). A local search template.

Great Britain: Elseiver Science Ltd.

TRITA -EECS-EX-2019:164

www.kth.se