COORDINATING SERVICES FOR
ACCESSING AND PROCESSING DATA IN
DYNAMIC ENVIRONMENTS
http://optimacs.imag.fr
2
DYNAMIC ENVIRONMENT
Consists of services, servers and devices that can be static or nomad
Data producers provide data on demand (e.g., online applications, Web-hosted DBMS) continuously – streams-- (e.g., messaging systems, mobile devices)
Data are hidden behind services export API through heterogeneous networks provide functions for retrieving and processing data
“modern data and services intensive systems” deployed in dynamic
environments
3
Give me five cinemas with available seats, located less than 3 Km from my current positionand that are showing 3 – 5 stars movies released within the last 10 days
QUERYING IN DYNAMIC ENVIRONMENTS
Is spatio-temporal or not Consumes on demand data or streams from static or nomad data services Is evaluated continuously and in batch
4
VISION: SERVICE COORDINATION FOR OPTIMALLY QUERYING DATA
geoLocate() dist([(x1,y1,z1),(x2,y2,z2)],3)
getfilms(3-5, 10)
onScreen(film) showLocation()
Give me five cinemas with available seats, located less than 3 Km from my current positionand that are showing 3 – 5 stars movies released within the last 10 days
⋈
filter(list, 5)
GetMyLocation
Look4films
filter(list,10 days)
OnScreen
Correlate
ComputeDistance
LocateTheatres
TemporalFilter Generate
Map
Filter
geoLocate() dist([(x1,y1,z1),(x2,y2,z2)],3)
getfilms(3-5, 10)
onScreen(film) showLocation()⋈
filter(list, 5)
filter(list,10 days)
VISION: SERVICE COORDINATION FOR OPTIMALLY QUERYING DATA
5
OnScreen
GetMyLocation
Look4films
Correlate
ComputeDistance
LocateTheatres
TemporalFilter Generate
Map
Filter
MINIMIZE INFRASTRUCTURE ACCESS COSTS
MINIMIZE PLATFORM ACCESS COSTS
MINIMIZE COORDINATION COSTS: TIME, BATTERY, ECONOMICAL
MAXIMIZE QOS: PERTINENCE, FRESHNESS, ACCURACY
6
ASPECTS TO CONSIDER
Data providers are services Export an API and are accessible through a lookup service Few information about data produced: pivot data model for exchanging data Autonomy QoS properties: pertinence (semantic, geographic, temporal, provenance)
Data consumers Express their data requirements (language) Consume data continuously or on demand Nomad/static
Execution context Ubiquitous Dynamic: resources availability change all the time Heterogeneous devices: different physical and computing capacities
7
Services coordination for evaluating hybrid queries
Combine service composition and query evaluation
Optimize hybrid queries according to quality of service criteria
Propose a testbed for validating query evaluation based on service coordination within « real » dynamic environments
No off-the-shelf DBMS for evaluating different types of queries
CHALLENGES
8
OBJECTIVES
Efficient and adaptable evaluation of hybrid queries in service oriented environments
Propose an adaptable hybrid query evaluation process
and associated mechanisms Propose QoS based optimization techniques for hybrid
queries Design and implement a benchmark and testbed for
dynamic environments
9
QUERY WORKFLOW
Expressed in a CQL-Like declarative language
Implemented by a service coordination: query workflow
IAAS
SAAS
PAAS Computingservices
Storage services
Data services
π
σ
⋈
σ
10
IMPLEMENTATION ISSUES
IAAS
PAAS SAAS
Query parser Query coordinationconstructor
Workflow engine Scheduler⟨outTuple(s)⟩
inputOp1()
CQL-Like expression
workflow
activity
COMPUTING SERVICES
DATA SERVICES
DEVICES
DATASPACE
MOBILEQ
HYPATIA
11
WHAT’S NEXT?
Optimization approach: multi-objective combinatory problem Problem expression on an inference engine State of the art of workflow optimization QoS measures computing and definition of multi-dimensional cost functions
Observation and QoS measures computing mechanism Extension of the testbed DATASPACE Testing of hybrid query evaluation with QoS
Benchmark definition for measuring large scale hybrid query evaluation Extend and validate scenarios Validate the benchmark
12
http://optimacs.imag.fr
COORDINATING SERVICES FOR ACCESSING AND PROCESSING DATA IN DYNAMIC ENVIRONMENTS
Gracias
13
DATA SERVICES
Discrete data or continuous data providers Export API and properties Methods are tagged with the result production rate
profile@facebook
coordinates@latitude
profile: email person<email,name,nickname,gender,age>
Suscribe: email coordinates(email,coor)