1

LYU0401 Location-LYU0401 Location-Based Multimedia Based Multimedia

Mobile ServiceMobile ServiceClarence FungClarence Fung

Tilen MaTilen MaSupervisor: Professor Michael LyuSupervisor: Professor Michael Lyu

Marker: Professor Alan LiewMarker: Professor Alan Liew

2

OutlineOutline

ObjectiveObjective Introduction to LocalizationIntroduction to Localization Area-based ProbabilityArea-based Probability Center of MassCenter of Mass Multimedia Guidance SystemMultimedia Guidance System ConclusionConclusion

3

ObjectiveObjective

To study different algorithms and To study different algorithms and techniques in localizationtechniques in localization

To find out a suitable localization To find out a suitable localization method for Location-Based Servicemethod for Location-Based Service

To develop an application for rapid To develop an application for rapid Location-Based Multimedia Service Location-Based Multimedia Service System DevelopmentSystem Development

4

Main Steps in Main Steps in LocalizationLocalization

Decide the Areas

Measure Signals at Decided Areas

Create a Training Set

Measure Signals at Current Position

Create a Testing Set

Apply Suitable Algorithm Return the most likely Area

5

Decide the AreasDecide the Areas

6

Collecting Training Set Collecting Training Set Data (I)Data (I)

In one particular area AIn one particular area Aii, we read a ser, we read a series of signal strengths (sies of signal strengths (sijkijk ) for a partic ) for a particular APular APj j with a constant time between swith a constant time between samplesamples

We estimate sWe estimate sij ij by averaging the series, by averaging the series, {s{sij1ij1, s, sij2ij2…, s…, sijoijo } }

7

Collecting Training Set Collecting Training Set Data (II)Data (II)

We read signals of all n APs, so we havWe read signals of all n APs, so we have the fingerprints at Ae the fingerprints at Aii

We read signals at all m areasWe read signals at all m areas

8

Collecting Testing Set Collecting Testing Set DataData

collects a set of received signal strengtcollects a set of received signal strengths when it is at certain locationhs when it is at certain location

similar to the fingerprints in the trainisimilar to the fingerprints in the training setng set

a set of average signal strengths from a set of average signal strengths from APsAPs

9

Data ProcessingData Processing

delete some access points that have delete some access points that have least contribution to localization least contribution to localization To shorten computation timeTo shorten computation time

input -92 dBm for missing signal input -92 dBm for missing signal strengthsstrengths

10

Training Set ExampleTraining Set ExamplePosition 1 2 3 4 5 6 7 8 9 10 11 12

AP MAC address Signal Strength (dBm)

00:02:2d:28:be:9e -70 -62 -58 -67 -73 -78 -83 -86 -84 -81 -78 -55

00:02:2d:28:be:5d -67 -59 -60 -71 -76 -79 -81 -86 -81 -83 -79 -52

00:60:1d:1e:43:9b -79 -87 -85 -84 -89 -80 -76 -77 -66 -63 -77 -90

00:0f:34:f3:60:40 -63 -69 -65 -74 -76 -72 -77 -84 -76 -74 -66 -79

00:02:2d:21:39:1f -92 -92 -82 -78 -82 -59 -78 -73 -83 -85 -82 -92

00:11:93:3d:6f:c0 -92 -92 -92 -90 -85 -86 -89 -88 -92 -92 -92 -92

00:0f:34:bb:df:20 -92 -92 -92 -89 -90 -82 -88 -88 -92 -92 -92 -92

11

Testing Set ExampleTesting Set Example

AP MAC address Signal Strength (dBm)

00:02:2d:28:be:9e -71

00:02:2d:28:be:5d -72

00:60:1d:1e:43:9b -89

00:0f:34:f3:60:40 -49

00:02:2d:21:39:1f -92

00:11:93:3d:6f:c0 -92

00:0f:34:bb:df:20 -92

12

Area-Based Probability Area-Based Probability (ABP)(ABP)

Advantages:Advantages: Area-based approachArea-based approach More mathematical approachMore mathematical approach Higher accuracyHigher accuracy

Approach: Approach: compute the likelihood of the testing set (Scompute the likelihood of the testing set (Stt) t) t

hat matches the fingerprint for each area (Shat matches the fingerprint for each area (Sii) )

13

Assumptions in ABPAssumptions in ABP

Signal received from different APs are Signal received from different APs are independent independent

For each AP, the sequence of RSS sFor each AP, the sequence of RSS s ijkijk, is , is modeled as a Gaussian distributionmodeled as a Gaussian distribution

14

Applying Bayes’ ruleApplying Bayes’ rule We compute the probability of being at We compute the probability of being at

different areas Adifferent areas Aii, on given the testing s, on given the testing set Set Stt

P(AP(Ai i |S|Stt) = P(S) = P(St t |A|Aii)* P(A)* P(Aii)/ P(S)/ P(Stt) (1)) (1) P(SP(Stt) is a constant) is a constant Assume the object is equally likely to be at any lAssume the object is equally likely to be at any l

ocation. P(Aocation. P(Aii) is a constant) is a constant

P(AP(Ai i |S|Stt) = c*P(S) = c*P(St t |A|Aii)) (2) (2)

15

Gaussian Distribution Gaussian Distribution We compute P(SWe compute P(St t ||

AAii) for every area ) for every area AAii ,i=1…m, using t ,i=1…m, using the Gaussian assuhe Gaussian assumptionmption

Integral of Normal Integral of Normal Function taking thFunction taking the interval as 1 e interval as 1

16

Error function erf(x)Error function erf(x) Express Integral of Express Integral of

Normal Function in Normal Function in terms of erfterms of erf

Approximate value Approximate value of erf by a seriesof erf by a series

17

Use of probabilityUse of probability

Discrete Space EstimationDiscrete Space Estimation Only fixed number of locations in the training set Only fixed number of locations in the training set

can be returnedcan be returned Eg. Return the area AEg. Return the area Aii with top probability with top probability

Continuous Space EstimationContinuous Space Estimation Continuous range of locations can be returnedContinuous range of locations can be returned Return locations may or may not be in the traininReturn locations may or may not be in the trainin

g setg set Eg. Finding Center of MassEg. Finding Center of Mass

18

Accuracy of Discrete Space Accuracy of Discrete Space EstimationEstimation

Default Default sample size sample size of testing of testing set = 4set = 4

80 testing 80 testing sets for each sets for each of the 12 of the 12 locations locations

8580

75 74

8575 71

87.5

52.5

70

87.5

6975.94

0

10

20

30

40

50

60

70

80

90

Percentage

1 2 3 4 5 6 7 8 9 10 11 12 Overall

Area

Accuracy at Different Locations

19

Continuous Space Continuous Space Estimation (CSE)Estimation (CSE)

Advantage:Advantage: Return locations may or may not be in Return locations may or may not be in

the training setthe training set Higher accuracyHigher accuracy Suitable for mobile applicationSuitable for mobile application

Two techniques:Two techniques: Center of MassCenter of Mass Time-AveragingTime-Averaging

20

Center of MassCenter of Mass

Assume n locationsAssume n locations Treat each location in the training Treat each location in the training

set as an objectset as an object Each object has a weight equals to Each object has a weight equals to

its probability densityits probability density Obtain Center of Mass of n objects Obtain Center of Mass of n objects

using their weighted positionsusing their weighted positions

21

Center of MassCenter of Mass

Let p(i) be the probability of a location xLet p(i) be the probability of a location xii, i=1,, i=1,2 …n2 …n

Let Y be the set of locations in 2D space and YLet Y be the set of locations in 2D space and Y(i) is the corresponding position of x(i) is the corresponding position of xii

The Center of Mass is given by:The Center of Mass is given by:

22

ExampleExample

23

Experiment on Center of Experiment on Center of MassMass

16 positions for the training set16 positions for the training set In every location, we use our system In every location, we use our system

to perform positioning for 100 times to perform positioning for 100 times Totally we get 1600 records Totally we get 1600 records

24

Experiment ResultExperiment Result

25

Positioning Accuracy at Location 0

0

2

4

6

8

10

12

14

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58

Distance Error

Frequency

26

Summary of statistics at all Summary of statistics at all the positionsthe positions

8393

7767

99

84838472

96

7683

909385

9785.125

01020304050

60708090

100

%

0 2 4 6 8 10 12 14 Overall

Position

Distribution of Estimated Points within 2 meters atdifferent position

27

9399

8877

10096929490100

9387

100100969994

01020304050

60708090

100

%

0 2 4 6 8 10 12 14 Overall

Position

Distribution of Estimated Points within 3 meters atdifferent position

28

Analysis of Experiment Analysis of Experiment Result Result

If the tolerance of error is 2 meters If the tolerance of error is 2 meters and 3 meters, the accuracy of our and 3 meters, the accuracy of our system is 85% and 94% respectivelysystem is 85% and 94% respectively

A few estimated points with a large A few estimated points with a large distance error (3m to 5m)distance error (3m to 5m) Inaccuracy in testing setInaccuracy in testing set Frustration in signal strengthFrustration in signal strength

29

Application SystemApplication System

In our project, we have implemented In our project, we have implemented two development toolstwo development tools Wi-Fi Location System (WLS)Wi-Fi Location System (WLS)

To develop Location-Based SystemTo develop Location-Based System Multimedia Guidance System (MGS)Multimedia Guidance System (MGS)

To develop Location-Based System with To develop Location-Based System with multimedia servicesmultimedia services

Developer can develop any Location-Developer can develop any Location-Based Multimedia System using our Based Multimedia System using our toolstools

30

Development Development EnvironmentEnvironment

Platform:Platform: Window CEWindow CE Window XP, 2000Window XP, 2000

Technology:Technology: IEEE 802.11bIEEE 802.11b

ToolsTools Embedded Visual C++ 4.0Embedded Visual C++ 4.0 Visual Studio .NET 2003Visual Studio .NET 2003

31

Wireless LAN Wireless LAN TerminologyTerminology

Media Access Control address (MAC Media Access Control address (MAC Address)Address) 48 bits long48 bits long unique hardware address unique hardware address e.g. 00:50:FC:2A:A9:C9e.g. 00:50:FC:2A:A9:C9

Service set identifier (SSID)Service set identifier (SSID) 32 character32 character Wireless LAN identifierWireless LAN identifier

Receive Signal Strength Indicator (RSSI) Receive Signal Strength Indicator (RSSI) signal strength signal strength unit is in dBm unit is in dBm

32

Wi-Fi Location System Wi-Fi Location System (WLS) (WLS)

Development Tool for Location-Based Development Tool for Location-Based SystemSystem

Simplify development stepsSimplify development steps Increase the efficiency and productivitIncrease the efficiency and productivit

y y It divides into 3 componentsIt divides into 3 components

Wi-Fi Signal Scanner (WSS)Wi-Fi Signal Scanner (WSS) Wi-Fi Data Processor (WDP)Wi-Fi Data Processor (WDP) Wi-Fi Location Detector (WLD)Wi-Fi Location Detector (WLD)

33

Wi-Fi Location SystemWi-Fi Location System Wi-Fi Signal ScannerWi-Fi Signal Scanner

To collect the signal strength received from To collect the signal strength received from access pointsaccess points

Wi-Fi Data ProcessorWi-Fi Data Processor To process collected dataTo process collected data To filter noise dataTo filter noise data

Wi-Fi Location DetectorWi-Fi Location Detector To detect the location in target place To detect the location in target place To show the detected position name and To show the detected position name and

corresponding position at Map Picture corresponding position at Map Picture

34

Multimedia Guidance Multimedia Guidance System (MGS)System (MGS)

WLS LimitationsWLS Limitations only provides Location-Based servicesonly provides Location-Based services does not support any multimedia does not support any multimedia

servicesservices Discrete Space EstimationsDiscrete Space Estimations

We create new development tools to We create new development tools to support multimedia servicessupport multimedia services

35

Multimedia Guidance Multimedia Guidance System (MGS)System (MGS)

Multimedia Guidance System is a Multimedia Guidance System is a development tool to support development tool to support multimedia servicemultimedia service

It consists of three componentsIt consists of three components Wi-Fi Signal Scanner (WSS)Wi-Fi Signal Scanner (WSS) Multimedia Guidance Processor (MGP)Multimedia Guidance Processor (MGP) Location-Based Multimedia Service Location-Based Multimedia Service

System (LBMSS)System (LBMSS)

36

Multimedia Guidance Multimedia Guidance SystemSystem

Deploying ProcedureDeploying Procedure

Processing Data and generating

LBSData

Collecting Signal

strength data

Deploying System with clients and

server application

37

Wi-Fi Signal Scanner Wi-Fi Signal Scanner (WSS) (WSS)

To collect the signal strength received To collect the signal strength received from access pointsfrom access points

The collected data received from WSS iThe collected data received from WSS is then processed by MGPs then processed by MGP

Same as WSS in WLSSame as WSS in WLS

38

Wi-Fi Signal ScannerWi-Fi Signal Scanner

MAC Address

SSID WEP Signal Strength

Number of received

data

Total Number of Signal Strength

Mean of Signal Strength

39

Multimedia Guidance Multimedia Guidance Processor (MGP)Processor (MGP)

To process and filter collected signal To process and filter collected signal strength datastrength data

To set client environment in LBMSSTo set client environment in LBMSS Server AddressServer Address Availability of Service Availability of Service Position InformationPosition Information

VideoVideo PicturePicture Position in Map PicturePosition in Map Picture

40

Multimedia Guidance Multimedia Guidance ProcessorProcessor

Data Processing Procedure Data Processing Procedure

Open the MGP

Set the project name, server path and map

picture path insetting

Add/Delete the target Position

Set the name, data, video, picture, position in the

map picture and point of interest

Filter the noise data

Setting Lines

Generate location-based

data

Select service for mobile

client

41

Multimedia Guidance Multimedia Guidance ProcessorProcessor

Position InformationAccess Point InformationPosition SettingService SettingInformation SectionEnvironment and Options Setting

42

Location-Based Multimedia Location-Based Multimedia Service System (LBMSS)Service System (LBMSS)

To provide multimedia service for To provide multimedia service for usersusers

It mainly consists of two partsIt mainly consists of two parts ClientClient ServerServer

Chat and Management ServerChat and Management Server Paint ServerPaint Server Media ServerMedia Server

43

Overall ArchitectureOverall Architecture

Mobile Client

Chat and Management ServerMedia ServerPaint Server

Mobile Client Mobile Client

44

Overall ArchitectureOverall Architecture

Techniques on Client/Server model in Techniques on Client/Server model in LBMSSLBMSS Socket Programming in .NETSocket Programming in .NET Multithread Model (AsyncCallback, Non-Multithread Model (AsyncCallback, Non-

Blocking Function)Blocking Function) Encode object variable into byte streamEncode object variable into byte stream Import Native Code (Visual C++ API) to Import Native Code (Visual C++ API) to

Managed Code (.NET API)Managed Code (.NET API) Control other program by CreateProcess() Control other program by CreateProcess()

APIAPI

45

Paint ServerPaint Server

To provide paint serviceTo provide paint service It has two main functionsIt has two main functions

Store Client’s ArtworkStore Client’s Artwork Print Artwork for ClientPrint Artwork for Client

46

Chat and Management Chat and Management ServerServer

To provide chat serviceTo provide chat service It shows the current position of all It shows the current position of all

clients on the map clients on the map

47

Media ServerMedia Server

To provide media To provide media serviceservice

It is made by It is made by existing server existing server architecture (e.g. architecture (e.g. HTTP, FTP, HTTP, FTP, Streaming Streaming Server)Server)

HTTP Server Streaming Server

Client

FTP Server

48

ClientClient

To provide multimedia service for To provide multimedia service for usersusers

Client program mainly has 4 servicesClient program mainly has 4 services GuideGuide PaintPaint ChatChat VideoVideo

49

ClientClient

Guide ServiceGuide Service Select DestinationSelect Destination Guide Line shows on Map Guide Line shows on Map

TabTab Implement by Shortest Implement by Shortest

Path AlgorithmPath Algorithm

50

Shortest Path in the Guide Shortest Path in the Guide Service Service

We define the nodes as the turning We define the nodes as the turning points or the ending of the corridors points or the ending of the corridors

51

add edges conneadd edges connecting two nodes cting two nodes

give a weight whgive a weight which is equal to thich is equal to the distance betwee distance between the two nodeen the two nodes s

1

5

2 6

3

4 7

8

81

116

38

76

200

200

154

43

52

add the source add the source node and the dnode and the destination nodestination nodee

add new edgesadd new edges

Use of DijkstraUse of Dijkstra’s algorithm’s algorithm

38

4 7

C

60

56

D110 90

1

5

2 6

81

38

76

200

154

43

53

Dijkstra’s algorithm Dijkstra’s algorithm 1. Set i=0, S1. Set i=0, S00= {u= {u00=s}, L(u=s}, L(u00)=0, and L(v)=infinity for )=0, and L(v)=infinity for

v <> uv <> u00. If |V| = 1 then stop, otherwise go to ste. If |V| = 1 then stop, otherwise go to step 2. p 2.

2. For each v in V\S2. For each v in V\Sii, replace L(v) by min{L(v), L(u, replace L(v) by min{L(v), L(uii)+d)+dvuvu}. If L(v) is replaced, put a label (L(v), u}. If L(v) is replaced, put a label (L(v), uii) ) on v. on v.

3. Find a vertex v which minimizes {L(v): v in V\S3. Find a vertex v which minimizes {L(v): v in V\Sii}, say u}, say uii+1. +1.

4. Let S4. Let Sii+1 = Si cup {u+1 = Si cup {uii+1}. +1}. 5. Replace i by i+1. If i=|V|-1 then stop, otherwise 5. Replace i by i+1. If i=|V|-1 then stop, otherwise

go to step 2. go to step 2.

54

ClientClient

Paint ServicePaint Service1.1. Get Picture from Get Picture from

Media ServerMedia Server

2.2. Draw on PictureDraw on Picture

3.3. Send Artwork to Paint Send Artwork to Paint ServerServer

55

ClientClient

Chat ServiceChat Service Message can be sent Message can be sent

both directionboth direction

i.e. Client to Server or i.e. Client to Server or Server to ClientServer to Client

56

ClientClient

Video ServiceVideo Service Play video by clicking Play video by clicking

“Play” Button“Play” Button Video is played by Video is played by

Windows Media Player Windows Media Player (WMP)(WMP)

WMP is controlled by WMP is controlled by CreateProcess() API in CreateProcess() API in Embedded Visual C++Embedded Visual C++

57

Comparison between Comparison between Tradition Method and MGS Tradition Method and MGS

MethodMethod Tradition MethodTradition Method

Studying the Technology Studying the Technology (1-2 week)(1-2 week) Software Design Software Design (2-3 week)(2-3 week) Architecture Design Architecture Design (2-3 week)(2-3 week) Algorithm Design Algorithm Design (1-2 week)(1-2 week)

MGS MethodMGS Method Wi-Fi Signal Scanner Wi-Fi Signal Scanner (1/2 day)(1/2 day) Multimedia Guidance Processor Multimedia Guidance Processor (2-3 days)(2-3 days) Location-Based Multimedia Service Location-Based Multimedia Service

System System (4-5 days)(4-5 days)

58

Comparison between Comparison between Tradition Method and MGS Tradition Method and MGS

MethodMethod Using MGS method, we can develop LoUsing MGS method, we can develop Lo

cation-Based Multimedia Service Systecation-Based Multimedia Service System in a short time.m in a short time.

This work can be done by non-professiThis work can be done by non-professionalsonals

It simplifies the development stepsIt simplifies the development steps

59

ConclusionConclusion

We are successful in achieving the goaWe are successful in achieving the goal of localizationl of localization

We have done experiments on accuracWe have done experiments on accuracy of algorithmy of algorithm

We have implemented two location-baWe have implemented two location-based development toolssed development tools Wi-Fi Location SystemWi-Fi Location System Multimedia Guidance SystemMultimedia Guidance System

60

Q&AQ&A

61

The ENDThe END