the data streaming paradigm and its use in fog architectures
TRANSCRIPT
Agenda
• Introduction:Fogarchitectures• Datastreamingbasics• Trade-offsandchallengesinFogstreaminganalysis• Conclusions• Bibliography
2ThedatastreamingparadigmanditsuseinFogarchitectures
Agenda
• Introduction:Fogarchitectures• Datastreamingbasics• Trade-offsandchallengesinFogstreaminganalysis• Conclusions• Bibliography
3ThedatastreamingparadigmanditsuseinFogarchitectures
Introduction:Fogarchitectures
AdvancedMetering Infrastructures(AMIs)[1,2,3,4]
VehicularNetworks(VNs)[5,6]
4ThedatastreamingparadigmanditsuseinFogarchitectures
AMIs VNs
• demand-response• scheduling [7]• micro-grids• detectionofmediumsizeblackouts [8]• detectionofnon technicallosses• ...
5
• autonomous driving• platooning• accidentdetection[9]• variabletolls[9]• congestionmonitoring [10]• ...
Accesstotheirdataà Increasedawareness,security,power-efficiency,...
ThedatastreamingparadigmanditsuseinFogarchitectures
(some)fundamentalquestions
• wheredoweprocessdata?• howdoweprocessdata?• howdoweenforcesecurityandprivacy?
6ThedatastreamingparadigmanditsuseinFogarchitectures
thewrong incompleteanswerweruntheanalysisinthecloud!
7
1. doestheinfrastructureallowforbillionsofreadingsperdaytobetransferredcontinuously?
2. thelatencyincurredwhiletransferringdata,doesthatunderminetheutilityoftheanalysis?
3. isitsecuretoconcentrateallthedatainasingleplace?[11]
4. isitsmarttogiveawayfine-graineddata?[12]
ThedatastreamingparadigmanditsuseinFogarchitectures
Asmallexampleofwhatfine-graineddatacanreveal...
8
source:PrivateMemoirsofaSmartMeter[12]
ThedatastreamingparadigmanditsuseinFogarchitectures
abetteranswerweleveragetheentireinfrastructure!
9
Traditionalanalysistechniquescannotaddressallthechallengesinthesesetups[13,14]
ThedatastreamingparadigmanditsuseinFogarchitectures
Fogarchitectures
10
source:FogComputingandItsRoleintheInternetofThings[15]
Characteristics[15]:1. edgelocation/location
awareness/lowlatency2. geographicaldistribution3. large-scale4. supportformobility5. real-timeinteractions6. predominanceofwireless7. heterogeneous8. interoperability/federation9. interactionwiththecloud
ThedatastreamingparadigmanditsuseinFogarchitectures
Fogarchitectures,applicationstrade-offsandchallenges
11
Sensor/EdgeDevices FogDevices CloudDevices
Cost + ++ +++1
Computationalpower + ++ +++
Inter-nodeheterogeneity +++ ++ +
Intra-node heterogeneity + + +++
Size +++ ++ +
Communication wireless wireless/wired wired
Volumeofdata millionsofsmallstreams2 thousands ofmediumaggregatedstreams manylargestreams
Evolutionovertime +++ ++ +3
1. depends onwhoownsthephysical hardware2. withexceptionslikee.g.Lidarsensors [16,17]3. Virtualenvironments canalsoevolve ”behindthescenes”
ThedatastreamingparadigmanditsuseinFogarchitectures
Agenda
• Introduction:Fogarchitectures• Datastreamingbasics• Trade-offsandchallengesinFogstreaminganalysis• Conclusions• Bibliography
12ThedatastreamingparadigmanditsuseinFogarchitectures
Motivation
Applications insensornetworks,cyber-physicalsystems:• largeandfluctuatingvolumesofdatagenerated
continuouslydemand for:• Continuous processingofdatastreams• Inareal-timefashion
Store-then-processisnotfeasible!!!
13ThedatastreamingparadigmanditsuseinFogarchitectures
MainMemory
Motivation
DBMSvs.DSMS
Disk
1 Data
QueryProcessing
3 Queryresults
2 Query
MainMemory
QueryProcessing
ContinuousQueryData Query
results
14ThedatastreamingparadigmanditsuseinFogarchitectures
Beforewestart...aboutdatastreamingandStreamProcessingEngines(SPEs)
15
Anincomplete,non-sortedlistofSPEs(cf.relatedworkin[18]):
time
BorealisThe Aurora Project
STanfordstREamdatAManager
NiagaraCQ
COUGAR
StreamCloud
Coveringallofthem/discussingwhichusecasesarebestforeachoneoutofscope...willfocusontheonesIworkwithandonexamplefromvehicularnetworks
ThedatastreamingparadigmanditsuseinFogarchitectures
datastream:unboundedsequenceoftuplessharingthesameschema
16
Example:vehicles’speedreports
time
Field Field
vehicleid text
time(secs) text
speed(Km/h) double
Xcoordinate double
Ycoordinate double
A 8:00 55.5 X1 Y1
Let’sassumeeachsource(e.g.,vehicle)producesanddeliversatimestampsortedstream
A 8:07 34.3 X3 Y3
A 8:03 70.3 X2 Y2
ThedatastreamingparadigmanditsuseinFogarchitectures
continuousquery(orsimplyquery):DirectedAcyclicGraph(DAG)ofstreamsandoperators
17
OP
OP
OP
OP OP
OP OP
sourceop(1+outstreams)
sinkop(1+instreams)
stream
op(1+in,1+outstreams)
ThedatastreamingparadigmanditsuseinFogarchitectures
datastreamingoperators
Twomaintypes:• Statelessoperators
• donotmaintainanystate• one-by-oneprocessing• iftheymaintainsomestate,suchstatedoesnotevolvedependingonthetuplesbeingprocessed
• Statefuloperators• maintainastatethatevolvesdependingonthetuplesbeingprocessed
• produceoutputtuplesthatdependonmultipleinputtuples
18
OP
OP
ThedatastreamingparadigmanditsuseinFogarchitectures
statelessoperators
19
Filter ...
Map
Union...
Filter/routetuplesbasedonone(ormore)conditions
Transformeachtuple
Mergemultiplestreams(withthesameschema)intoone
ThedatastreamingparadigmanditsuseinFogarchitectures
statelessoperators
20
Filter ...
Map
Union...
source:http://storm.apache.org/releases/2.0.0-SNAPSHOT/Trident-tutorial.html
ThedatastreamingparadigmanditsuseinFogarchitectures
statelessoperators
21
Filter ...
Map
Union...
source:https://ci.apache.org/projects/flink/flink-docs-release-1.0/apis/streaming/index.html
ThedatastreamingparadigmanditsuseinFogarchitectures
statelessoperators
22
Filter ...
Map
Union...
source:http://spark.apache.org/docs/latest/streaming-programming-guide.html#transformations-on-dstreams
ThedatastreamingparadigmanditsuseinFogarchitectures
statefuloperators
23
Aggregateinformationfrommultipletuples(e.g.,max,min,sum,...)
Jointuplescomingfrom2streamsgivenacertainpredicate
Aggregate
Join
ThedatastreamingparadigmanditsuseinFogarchitectures
statefuloperators
24
source:http://storm.apache.org/releases/2.0.0-SNAPSHOT/Trident-tutorial.html
source:http://spark.apache.org/docs/latest/streaming-programming-guide.html#transformations-on-dstreams
source:http://spark.apache.org/docs/latest/streaming-programming-guide.html#transformations-on-dstreams
ThedatastreamingparadigmanditsuseinFogarchitectures
Waitamoment!
ifstreamsareunbounded,howcanweaggregateorjoin?
25ThedatastreamingparadigmanditsuseinFogarchitectures
windows andstatefulanalysis[18]
Statefuloperationsaredoneoverwindows:• Time-based(e.g.,tuplesinthelast10minutes)• Tuple-based(e.g.,giventhelast50tuples)
26
time[8:00,9:00)
[8:20,9:20)
[8:40,9:40)
Usuallyapplicationsrelyontime-basedslidingwindows
ThedatastreamingparadigmanditsuseinFogarchitectures
time-basedslidingwindowaggregation(count)
27
Counter:4
time[8:00,9:00)
8:05 8:15 8:22 8:45 9:05
Output:4
Counter:1Counter:2
Counter:3
Counter:3
time
8:05 8:15 8:22 8:45 9:05
[8:20,9:20)
weassumedeachsourceproducesanddeliversatimestampsortedstream!Whathappensifthisisnotthecase?
ThedatastreamingparadigmanditsuseinFogarchitectures
time-basedslidingwindowjoining
28
t1
t2
t3
t4
t1
t2
t3
t4
R S
Slidingwindow Window
size WSWSWR
PredicateP
ThedatastreamingparadigmanditsuseinFogarchitectures
30
basicoperatorsanduser-definedoperators
Besidesasetofbasicoperators,SPEsusuallyallowtheusertodefinead-hocoperators(e.g.,whenexistingaggregationarenotenough)
ThedatastreamingparadigmanditsuseinFogarchitectures
samplequery
Foreachvehicle,raiseanalertifthespeedofthelatestreportismorethan2timeshigherthanitsaveragespeedinthelast30days.
31
time
A 8:00 55.5 X1 Y1 A 8:07 34.3 X3 Y3
A 8:03 70.3 X2 Y2
ThedatastreamingparadigmanditsuseinFogarchitectures
32
Removeunusedfields
Map
Field
vehicleid
time(secs)
speed(Km/h)
Xcoordinate
Ycoordinate
Field
vehicleid
time(secs)
speed(Km/h)
Computeaveragespeedforeach
vehicleduring thelast30days
Aggregate
Fieldvehicleid
time(secs)
avgspeed(Km/h)
Join
Checkcondition
Filter
Field
vehicleid
time(secs)
speed(Km/h)
Joinonvehicleid
Fieldvehicleid
time(secs)
avgspeed(Km/h)
speed(Km/h)
samplequery
ThedatastreamingparadigmanditsuseinFogarchitectures
33
M A J F
samplequery
Notice:• thesamesemanticscanbedefinedinseveralways(usingdifferentoperatorsandcomposingthemindifferentways)
• Usingmanybasicbuildingblockscaneasethetaskofdistributingandparallelizingtheanalysis(moreinthefollowing...)
ThedatastreamingparadigmanditsuseinFogarchitectures
Agenda
• Introduction:Fogarchitectures• Datastreamingbasics• Trade-offsandchallengesinFogstreaminganalysis• Conclusions• Bibliography
34ThedatastreamingparadigmanditsuseinFogarchitectures
1. Distributeddeployment2. Paralleldeployment3. Orderinganddeterminism4. Shared-nothingvsshared-memoryparallelism5. Loadbalancing6. Elasticity7. Faulttolerance8. Datasharing(differentialprivacy)
35
8 aspects/usecasestodiscusschallenges(and opportunities)
ThedatastreamingparadigmanditsuseinFogarchitectures
Beforewestart...
36
Followingexamplesarefromvehicularnetworks
Road-sideunitRSU Vehicle
Server
ThedatastreamingparadigmanditsuseinFogarchitectures
1- Distributeddeployment– wheretoplaceagivenoperator?[19]
37
M
1. Whathardwareisavailableateachpossibledeploymentlocation?
2. Whatdataisavailableateachpossibledeploymentlocation,dependingon1. theinfrastructure?2. itssecurityregulations?3. itsprivacyregulations?
?
ThedatastreamingparadigmanditsuseinFogarchitectures
1- Distributeddeployment– wheretoplaceagivenoperator?
38
M1. Lowlatency(ifalltherequireddataislocally
accessible)2. Accesstoalllocaldata(partsofwhichmight
notbereportedtoRSUs/Servers)[4]3. Embeddeddevices/limitedcomputational
power[20]
ThedatastreamingparadigmanditsuseinFogarchitectures
1- Distributeddeployment– wheretoplaceagivenoperator?
39
M1. Accesstomultiplevehicles/highmobility2. Accesstosmallersetsofdata/partialviewof
thedata[22]3. Embeddeddevices/highercomputational
power4. Wireless/wiredcommunication
ThedatastreamingparadigmanditsuseinFogarchitectures
1- Distributeddeployment– wheretoplaceagivenoperator?
40
1. Globalviewofthedata(whenpossible[22])2. Accesstoaggregateddata3. Highcomputationalpower
ThedatastreamingparadigmanditsuseinFogarchitectures
M
2- Paralleldeployment– howdoweparallelize theanalysis?
41
Ifaparalleloperator(e.g.,M)feedsmultipledownstreamparalleloperators(e.g.,A),howdoweroutetuples? [23,24]
M
M A
A
ThedatastreamingparadigmanditsuseinFogarchitectures
2- Paralleldeployment– howdoweparallelize theanalysis?
42
M
M A
A
A 8:00 55.5 X1 Y1
A 8:07 34.3 X3 Y3
A 8:03 70.3 X2 Y2
A 8:00 55.5 X1 Y1
A 8:07 34.3 X3 Y3
A 8:03 70.3 X2 Y2
Whencomputinge.g.per-vehiclestatistics,weneedtomakesurethatallthetuplesreferringtothesamevehiclesaresenttothesameaggregateinstance
ThedatastreamingparadigmanditsuseinFogarchitectures
• Dependingonthestatefuloperatorsemantic:• Partitioninputstreamintobuckets• Eachbucketisprocessedby1node
• #buckets>>#nodes
43
2- Paralleldeployment– howdoweparallelize theanalysis?
ThedatastreamingparadigmanditsuseinFogarchitectures
• Dependingonthestatefuloperatorsemantic:• Partitioninputstreamintobuckets• Eachbucketisprocessedby1node
• #buckets>>#nodes
Keysdomain
Agg1 Agg2 Agg3A
D
E
B
C F
44
2- Paralleldeployment– howdoweparallelize theanalysis?
ThedatastreamingparadigmanditsuseinFogarchitectures
• Dependingonthestatefuloperatorsemantic:• Partitioninputstreamintobuckets• Eachbucketisprocessedby1node
• #buckets>>#nodes
Keysdomain
Agg1 Agg2 Agg3A
D
E
B
C F
45
2- Paralleldeployment– howdoweparallelize theanalysis?
ThedatastreamingparadigmanditsuseinFogarchitectures
2- Paralleldeployment– howdoweparallelize theanalysis?
46
Howcanweparallelizetheanalysisofe.g.anaggregatewhentheoperationsarenotperformed fordisjointpartitionsofastream(e.g.,forallvehiclesratherthanforeachvehicle?)
M
M A
A
ThedatastreamingparadigmanditsuseinFogarchitectures
47
2- Paralleldeployment– howdoweparallelize theanalysis?
ThedatastreamingparadigmanditsuseinFogarchitectures
48
M
M A
A
A 8:00 55.5 X1 Y1
A 8:07 34.3 X3 Y3
A 8:03 70.3 X2 Y2
A 8:00 55.5 X1 Y1
A 8:07 34.3 X3 Y3
A 8:03 70.3 X2 Y2
Whatiftuplewithtimestamp8:00arrivesaftertuplewithtimestamp8:07?
3– Orderinganddeterminism
ThedatastreamingparadigmanditsuseinFogarchitectures
49
3– Orderinganddeterminism
iftuplesthataggregatedtogetherarearbitrarilyinterleaved,howtoweprovidecorrectresults?
time[8:00,9:00)
8:05 8:15 8:22 8:45 9:05
Output:4
time[8:00,9:00)
8:05 8:15 8:22 8:45 9:058:55
ThedatastreamingparadigmanditsuseinFogarchitectures
50
3– Orderinganddeterminism– merge-sorting[25]&buffering [26,27]
addTuple(tuple,sourceID)allows a tuple from sourceID to be merged by ScaleGate in the resulting timestamp-sorted stream of ready tuples.
getNextReadyTuple(readerID)provides to readerID the next earliest ready tuple that has not been yet consumed by the former.
https://github.com/dcs-chalmers/ScaleGate_Java
ThedatastreamingparadigmanditsuseinFogarchitectures
4– shared-nothingvs.shared-memoryparallelism[28]
51
M
M
...
A
AJ
...
Howtotakeadvantageofmulti-corearchitectures?
Howtoboostinter-nodeparallelismandintra-nodeparallelism?
ThedatastreamingparadigmanditsuseinFogarchitectures
Shared-nothingparallelstreamjoin
ProdR
ProdS
PU1
PU2
PUN
…
ConsAdd tupletoPUi S
Add tupletoPUi R
Consumeresults
Pickthenextreadytuplet:1. comparetwithalltuples inoppositewindowgivenP2. addttoitswindow3. removestaletuples fromt’swindow
ChoseaPU
ChoseaPU
Takethenextreadyoutput tuple
Scalability
Highthroughput
Lowlatency
Disjointparallelism
Skewresilience
Determinism
52
Merge
4– shared-nothingvs.shared-memoryparallelism
ThedatastreamingparadigmanditsuseinFogarchitectures
ProdR
ProdS
PU1
PU2
PUN
…53
enqueue()dequeue()
ConsMerge
Shared-nothingparallelstreamjoin
4– shared-nothingvs.shared-memoryparallelism
ThedatastreamingparadigmanditsuseinFogarchitectures
ProdR
ProdS
PU1
PU2
PUN
… ConsAdd tupleSGin
Add tupleSGin
Getnextreadyoutput tuplefromSGout
Getnextreadyinput tuplefromSGin1. comparetwithalltuples inoppositewindowgivenP2. addttoitswindowinaround-robinfashion3. removestaletuples fromt’swindow
54
SGin SGout
StepsforPU
Shared-memoryparallelstreamjoin
4– shared-nothingvs.shared-memoryparallelism
ThedatastreamingparadigmanditsuseinFogarchitectures
55
t1
t2
R S
WR
t3
t4
R S
t4
t1WR
R S
t4
t2
WR
R S
t4
WR
t3
Sequentialstreamjoin:
ScaleJoinwith3PUs:
Shared-memoryparallelstreamjoin
4– shared-nothingvs.shared-memoryparallelism
ThedatastreamingparadigmanditsuseinFogarchitectures
ProdR
ProdS
PU1
PU2
PUN
… ConsAdd tupleSGin
Add tupleSGin
Getnextreadyoutput tuplefromSGout
56
SGin SGout
Scalability
Highthroughput
Lowlatency
Disjointparallelism
Skewresilience
Determinism
ProdS
ProdS
ProdR
GetnextreadyinputtuplefromSGin1. comparetwithalltuplesinoppositewindowgivenP2. addttoitswindow inaroundrobinfashion3. removestaletuplesfromt’swindow
StepsforPUi
Shared-memoryparallelstreamjoin
4– shared-nothingvs.shared-memoryparallelism
ThedatastreamingparadigmanditsuseinFogarchitectures
5– loadbalancing&statetransfer[23,29]
57
IfweshifttheprocessingofacertainsubsetoftuplesfromnodeAtonodeB,howdotransferitspreviousstate?
MA
MA
ThedatastreamingparadigmanditsuseinFogarchitectures
Statetransferchallengingforstatefuloperators
A B
time
58
5– loadbalancing&statetransfer
ThedatastreamingparadigmanditsuseinFogarchitectures
WindowRecreationProtocol[23]
A B
time
A
A
B
SendtoA
SendtoB
59
5– loadbalancing&statetransfer
+ Nocommunicationbetweennodes- Completiontimeproportionaltowindowsize
ThedatastreamingparadigmanditsuseinFogarchitectures
StateRecreationProtocol
A B
time
B
B
B
CopytoB
60
TuplesreferringtocallerA
5– loadbalancing&statetransfer
+Minimizescompletiontime- Communicationbetweennodes
ThedatastreamingparadigmanditsuseinFogarchitectures
6– elasticity[23,30]
61
How/whentoprovisionordecommissionnewresourcesdependingontheanalysis’costsfluctuations?
J
ThedatastreamingparadigmanditsuseinFogarchitectures
J J
J
Targetutilizationthreshold
Upperutilizationthreshold
Lowerutilizationthreshold
0%
100%
CPUconsumptionMemoryLatency...
………
OP OP OPOP OP
Elasticityandloadbalancingactionsonper-subclusterbasis
62
6– elasticity[23]
ThedatastreamingparadigmanditsuseinFogarchitectures
63
6– elasticity
• Howmanyresourcesshouldbeprovisioned/decommissioned?[23]
• Howtomixloadbalancingandprovisioning/decommissioning?[23]
• Howtochosewhichresourcestoprovision/decommission?[30]
• Howtopredictwhenmore/lessresourceswillbeneeded?[30]
ThedatastreamingparadigmanditsuseinFogarchitectures
Load-awareprovisioning
1 2 4 7 11 2617
64
6– elasticity[23]
ThedatastreamingparadigmanditsuseinFogarchitectures
7– faulttolerance[relatedworkin18,31,32]
65
Howtoreplaceafailednodeminimizingrecoverytime(makingittransparenttotheenduser)?
ThedatastreamingparadigmanditsuseinFogarchitectures
J
J
J
J
J
J
Activestandby
Primary
Replica
AM
A
66
Cost:
RecoveryTime:
7– faulttolerance
ThedatastreamingparadigmanditsuseinFogarchitectures
Passivestandby
Primary
Replica
AM
APeriodic
checkpoints
67
Cost:
RecoveryTime:
7– faulttolerance
ThedatastreamingparadigmanditsuseinFogarchitectures
Passivestandby
Primary
AM
Periodiccheckpoints
68
Cost:
RecoveryTime:
Disk
7– faulttolerance
ThedatastreamingparadigmanditsuseinFogarchitectures
UpstreamBackup
AM Buffer
69
Cost:
RecoveryTime:
7– faulttolerance
ThedatastreamingparadigmanditsuseinFogarchitectures
FaulttoleranceandelasticitycanalsobeintegratedtogetherinsolutionthatexternalizethemanagementofstatefromtheSPE[32]
70
7– faulttolerance
ThedatastreamingparadigmanditsuseinFogarchitectures
8 – datasharing(differentialprivacy)[2,33,34,35]
71
Howtopreventprivacyleaks?
Supposeweareinterestedinpublishingvehicles’averagespeedoverawindowofonehour...
WecouldaggregatebyRSU!
ThedatastreamingparadigmanditsuseinFogarchitectures
8 – datasharing(differentialprivacy)
72
Waitamoment!what ifasinglevehicleisconnectedtoacertainRSU?
Whetheracertainmechanismpreservesornottheprivacyoftheunderlyingdatadependsontheknowledgeoftheadversary
Differentialprivacyassumestheworstcasescenario!
ThedatastreamingparadigmanditsuseinFogarchitectures
73
8 – datasharing(differentialprivacy)
time[8:00,9:00)
22.3Km/h 10Km/h ? 15Km/h
avgspeed:30Km/h
withoutdifferentialprivacy,ifknowsthespeedof,and
andtheavgspeed,thenitcanfindoutthespeedof
ThedatastreamingparadigmanditsuseinFogarchitectures
74
8 – datasharing(differentialprivacy)
time[8:00,9:00)
22.3Km/h 10Km/h ? 15Km/h
avgspeed:30Km/h
withdifferentialprivacy,ifknowsthespeedof,and,itdoes
gainextraknowledgefromtheavgspeedinordertofindthespeedof
+noise(calibratedfromprobabilisticdistribution) sothatwhetherparticipatesornot,theresultswillbenearlythesame
ThedatastreamingparadigmanditsuseinFogarchitectures
Agenda
• Introduction:Fogarchitectures• Datastreamingbasics• Trade-offsandchallengesinFogstreaminganalysis• Conclusions• Bibliography
76ThedatastreamingparadigmanditsuseinFogarchitectures
Introduction:Fogarchitectures
77
• wheredoweprocessdata?• howdoweprocessdata?• howdoweenforcesecurityandprivacy?
...cannotreallyanswertothesequestionsindependently...
ThedatastreamingparadigmanditsuseinFogarchitectures
Fogarchitectures,applicationstrade-offsandchallenges
78
Sensor/EdgeDevices FogDevices CloudDevices
Cost + ++ +++
Computationalpower + ++ +++
Inter-nodeheterogeneity +++ ++ +
Intra-node heterogeneity + + +++
Size +++ ++ +
Communication wireless wireless/wired wired
Volumeofdata millionsofsmallstreams thousands ofmediumaggregatedstreams manylargestreams
Evolutionovertime +++ ++ +
ThedatastreamingparadigmanditsuseinFogarchitectures
Agenda
• Introduction:Fogarchitectures• Datastreamingbasics• Trade-offsandchallengesinFogstreaminganalysis• Conclusions• Bibliography
79ThedatastreamingparadigmanditsuseinFogarchitectures
Bibliography1. Zhou,Jiazhen,RoseQingyang Hu,andYiQian."Scalabledistributedcommunicationarchitectures tosupportadvanced
metering infrastructureinsmartgrid."IEEETransactionsonParallelandDistributedSystems23.9(2012):1632-1642.2. Gulisano,Vincenzo,etal."BES:DifferentiallyPrivateandDistributedEventAggregationinAdvancedMeteringInfrastructures."
Proceedingsofthe2ndACMInternationalWorkshoponCyber-PhysicalSystemSecurity.ACM,2016.3. Gulisano,Vincenzo,MagnusAlmgren,andMarinaPapatriantafilou."Onlineandscalabledatavalidation inadvancedmetering
infrastructures."IEEEPESInnovativeSmartGridTechnologies,Europe.IEEE,2014.4. Gulisano,Vincenzo,MagnusAlmgren,andMarinaPapatriantafilou."METIS:atwo-tierintrusiondetectionsystemforadvanced
metering infrastructures."InternationalConferenceonSecurityandPrivacyinCommunicationSystems.Springer InternationalPublishing,2014.
5. Yousefi,Saleh,MahmoudSiadat Mousavi,andMahmoodFathy."Vehicularadhocnetworks(VANETs):challengesandperspectives."20066thInternationalConferenceonITSTelecommunications. IEEE,2006.
6. ElZarki,Magda,etal."Securityissuesinafuturevehicularnetwork."EuropeanWireless. Vol.2.2002.7. Georgiadis,Giorgos,andMarinaPapatriantafilou."Dealing withstoragewithoutforecastsinsmartgrids:Problem
transformationandonlineschedulingalgorithm."Proceedingsofthe29thAnnualACMSymposiumonAppliedComputing.ACM,2014.
8. Fu,Zhang,etal."Onlinetemporal-spatialanalysisfordetectionofcritical eventsinCyber-PhysicalSystems."BigData(BigData),2014IEEEInternationalConferenceon.IEEE,2014.
ThedatastreamingparadigmanditsuseinFogarchitectures 80
Bibliography9. Arasu,Arvind, etal."Linearroad:astreamdatamanagementbenchmark."ProceedingsoftheThirtieth
international conferenceonVerylargedatabases-Volume30.VLDBEndowment,2004.10. Lv,Yisheng,etal."Trafficflowpredictionwithbigdata:adeeplearningapproach."IEEETransactionson
IntelligentTransportationSystems16.2(2015):865-873.11. Grochocki,David,etal."AMIthreats,intrusion detectionrequirementsanddeployment
recommendations."SmartGridCommunications (SmartGridComm), 2012IEEEThirdInternationalConferenceon.IEEE,2012.
12. Molina-Markham,Andrés,etal."Privatememoirsofasmartmeter."Proceedingsofthe2ndACMworkshoponembedded sensingsystemsforenergy-efficiencyinbuilding. ACM,2010.
13. Gulisano,Vincenzo,etal."Streamcloud:Alargescaledatastreamingsystem."DistributedComputingSystems(ICDCS),2010IEEE30thInternationalConferenceon.IEEE,2010.
14. Stonebraker,Michael,Uǧur Çetintemel,andStanZdonik."The8requirementsofreal-timestreamprocessing."ACMSIGMODRecord34.4(2005):42-47.
15. Bonomi,Flavio,etal."Fogcomputing anditsroleintheinternetofthings."Proceedingsof thefirsteditionoftheMCCworkshoponMobilecloudcomputing.ACM,2012.
16. Himmelsbach,Michael,etal."LIDAR-based3Dobjectperception."Proceedingsof1stinternationalworkshoponcognition fortechnicalsystems.Vol.1.2008.
ThedatastreamingparadigmanditsuseinFogarchitectures 81
Bibliography17. Geiger,Andreas,etal."Visionmeetsrobotics:TheKITTIdataset."TheInternationalJournalofRoboticsResearch(2013):
0278364913491297.18. Gulisano,VincenzoMassimiliano. StreamCloud:AnElasticParallel-DistributedStreamProcessingEngine.Diss.Informatica,
2012.19. Cardellini, Valeria,etal."Optimaloperatorplacement fordistributedstreamprocessingapplications."Proceedingsofthe10th
ACMInternationalConferenceonDistributedandEvent-basedSystems.ACM,2016.20. Costache,Stefania,etal."UnderstandingtheData-ProcessingChallenges inIntelligentVehicularSystems."Proceedingsofthe
2016IEEEIntelligentVehiclesSymposium(IV16).21. Cormode,Graham."Thecontinuousdistributedmonitoringmodel."ACMSIGMODRecord42.1(2013):5-14.22. Giatrakos,Nikos,AntoniosDeligiannakis, andMinosGarofalakis."ScalableApproximateQueryTrackingoverHighlyDistributed
DataStreams."Proceedingsofthe2016InternationalConferenceonManagementofData.ACM,2016.23. Gulisano,Vincenzo,etal."Streamcloud:Anelasticandscalabledatastreamingsystem."IEEETransactionsonParalleland
DistributedSystems23.12(2012):2351-2365.24. Shah,Mehul A.,etal."Flux:Anadaptivepartitioningoperatorforcontinuousquerysystems."DataEngineering,2003.
Proceedings.19thInternationalConferenceon.IEEE,2003.
ThedatastreamingparadigmanditsuseinFogarchitectures 82
Bibliography25. Cederman,Daniel,etal."Briefannouncement:concurrentdatastructuresforefficientstreamingaggregation."Proceedingsof
the26thACMsymposiumonParallelism inalgorithmsandarchitectures.ACM,2014.26. Ji,Yuanzhen,etal."Quality-drivenprocessingofslidingwindowaggregatesoverout-of-orderdatastreams."Proceedingsofthe
9thACMInternationalConferenceonDistributedEvent-BasedSystems.ACM,2015.27. Ji,Yuanzhen,etal."Quality-drivendisorderhandlingforconcurrentwindowedstreamquerieswithsharedoperators."
Proceedingsofthe10thACMInternationalConferenceonDistributedandEvent-basedSystems.ACM,2016.28. Gulisano,Vincenzo,etal."Scalejoin:Adeterministic,disjoint-parallelandskew-resilient streamjoin."BigData(BigData),2015
IEEEInternationalConferenceon.IEEE,2015.29. Ottenwälder,Beate,etal."MigCEP:operatormigrationformobilitydrivendistributedcomplexeventprocessing."Proceedings
ofthe7thACMinternational conferenceonDistributedevent-basedsystems.ACM,2013.30. DeMatteis,Tiziano,andGabrieleMencagli. "Keepcalmandreactwithforesight:strategiesforlow-latencyandenergy-efficient
elasticdatastreamprocessing."Proceedingsofthe21stACMSIGPLANSymposiumonPrinciplesandPracticeofParallelProgramming.ACM,2016.
31. Balazinska,Magdalena,etal."Fault-tolerance intheBorealisdistributedstreamprocessingsystem."ACMTransactionsonDatabaseSystems(TODS)33.1(2008):3.
32. CastroFernandez,Raul,etal."Integratingscaleoutandfaulttolerance instreamprocessingusingoperatorstatemanagement."Proceedingsofthe2013ACMSIGMODinternational conferenceonManagementofdata.ACM,2013.
ThedatastreamingparadigmanditsuseinFogarchitectures 83
Bibliography
33. Dwork,Cynthia."Differentialprivacy:Asurveyofresults."InternationalConferenceonTheoryandApplicationsofModelsofComputation.SpringerBerlinHeidelberg,2008.
34. Dwork,Cynthia,etal."Differentialprivacyundercontinualobservation."Proceedingsoftheforty-secondACMsymposiumonTheoryofcomputing.ACM,2010.
35. Kargl,Frank,ArikFriedman,andRoksana Boreli."Differentialprivacyinintelligenttransportationsystems."ProceedingsofthesixthACMconferenceonSecurityandprivacyinwirelessandmobilenetworks.ACM,2013.
ThedatastreamingparadigmanditsuseinFogarchitectures 84