COORDINATING SERVICES FOR

ACCESSING AND PROCESSING DATA IN

DYNAMIC ENVIRONMENTS

http://optimacs.imag.fr

2

DYNAMIC ENVIRONMENT

Consists of services, servers and devices that can be static or nomad

Data producers provide data on demand (e.g., online applications, Web-hosted DBMS) continuously – streams-- (e.g., messaging systems, mobile devices)

Data are hidden behind services export API through heterogeneous networks provide functions for retrieving and processing data

“modern data and services intensive systems” deployed in dynamic

environments

3

Give me five cinemas with available seats, located less than 3 Km from my current positionand that are showing 3 – 5 stars movies released within the last 10 days

QUERYING IN DYNAMIC ENVIRONMENTS

Is spatio-temporal or not Consumes on demand data or streams from static or nomad data services Is evaluated continuously and in batch

4

VISION: SERVICE COORDINATION FOR OPTIMALLY QUERYING DATA

geoLocate() dist([(x1,y1,z1),(x2,y2,z2)],3)

getfilms(3-5, 10)

onScreen(film) showLocation()

Give me five cinemas with available seats, located less than 3 Km from my current positionand that are showing 3 – 5 stars movies released within the last 10 days

⋈

filter(list, 5)

GetMyLocation

Look4films

filter(list,10 days)

OnScreen

Correlate

ComputeDistance

LocateTheatres

TemporalFilter Generate

Map

Filter

geoLocate() dist([(x1,y1,z1),(x2,y2,z2)],3)

getfilms(3-5, 10)

onScreen(film) showLocation()⋈

filter(list, 5)

filter(list,10 days)

VISION: SERVICE COORDINATION FOR OPTIMALLY QUERYING DATA

5

OnScreen

GetMyLocation

Look4films

Correlate

ComputeDistance

LocateTheatres

TemporalFilter Generate

Map

Filter

MINIMIZE INFRASTRUCTURE ACCESS COSTS

MINIMIZE PLATFORM ACCESS COSTS

MINIMIZE COORDINATION COSTS: TIME, BATTERY, ECONOMICAL

MAXIMIZE QOS: PERTINENCE, FRESHNESS, ACCURACY

6

ASPECTS TO CONSIDER

Data providers are services Export an API and are accessible through a lookup service Few information about data produced: pivot data model for exchanging data Autonomy QoS properties: pertinence (semantic, geographic, temporal, provenance)

Data consumers Express their data requirements (language) Consume data continuously or on demand Nomad/static

Execution context Ubiquitous Dynamic: resources availability change all the time Heterogeneous devices: different physical and computing capacities

7

Services coordination for evaluating hybrid queries

Combine service composition and query evaluation

Optimize hybrid queries according to quality of service criteria

Propose a testbed for validating query evaluation based on service coordination within « real » dynamic environments

No off-the-shelf DBMS for evaluating different types of queries

CHALLENGES

8

OBJECTIVES

Efficient and adaptable evaluation of hybrid queries in service oriented environments

Propose an adaptable hybrid query evaluation process

and associated mechanisms Propose QoS based optimization techniques for hybrid

queries Design and implement a benchmark and testbed for

dynamic environments

9

QUERY WORKFLOW

Expressed in a CQL-Like declarative language

Implemented by a service coordination: query workflow

IAAS

SAAS

PAAS Computingservices

Storage services

Data services

π

σ

⋈

σ

10

IMPLEMENTATION ISSUES

IAAS

PAAS SAAS

Query parser Query coordinationconstructor

Workflow engine Scheduler⟨outTuple(s)⟩

inputOp1()

CQL-Like expression

workflow

activity

COMPUTING SERVICES

DATA SERVICES

DEVICES

DATASPACE

MOBILEQ

HYPATIA

11

WHAT’S NEXT?

Optimization approach: multi-objective combinatory problem Problem expression on an inference engine State of the art of workflow optimization QoS measures computing and definition of multi-dimensional cost functions

Observation and QoS measures computing mechanism Extension of the testbed DATASPACE Testing of hybrid query evaluation with QoS

Benchmark definition for measuring large scale hybrid query evaluation Extend and validate scenarios Validate the benchmark

12

http://optimacs.imag.fr

COORDINATING SERVICES FOR ACCESSING AND PROCESSING DATA IN DYNAMIC ENVIRONMENTS

Gracias

13

DATA SERVICES

Discrete data or continuous data providers Export API and properties Methods are tagged with the result production rate

profile@facebook

coordinates@latitude

profile: email person<email,name,nickname,gender,age>

Suscribe: email coordinates(email,coor)

14

COMPUTING SERVICES

Simple: only one service method call executed within the activity

Composite: multiple service methods calls in activities organized as a workflow

Sim

ilarit

y

Wordnet

Aggregation

Caching HT

Search engine