a design of distributed storage system over http for collecting sensor data
TRANSCRIPT
![Page 1: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/1.jpg)
1
A Design of Distributed Storage System
over HTTP for Collecting Sensor Data
Sayed Ahmad NaweedSupervisor: Dr. Yokoyama Teruaki
4th June 2016
![Page 2: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/2.jpg)
2
Environmental Monitoring
Transportation
Industry
Weather
Military
Medical
Intelligent Transportation
System
Background
![Page 3: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/3.jpg)
3
Application of Sensor Technology
Environmental Monitoring
Transportation
Industry
Weather
Military
Medical
![Page 4: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/4.jpg)
4
Collecting Sensor Data
![Page 5: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/5.jpg)
5
Collecting Sensor Data
![Page 6: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/6.jpg)
6
Collecting Sensor Data
![Page 7: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/7.jpg)
7
Collecting Sensor Data
Data Collection
![Page 8: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/8.jpg)
8
Afghanistan IT Infrastructure
Afghanistan National Data Center (ANDC)
e-Government
e-Commerce
m-Government
Cloud Computing
Sources: http://andc.gov.af/en/page/1098
![Page 9: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/9.jpg)
9
Possible Image of Sensor Network in Afghanistan
Sensors
ANDC Location
App
App
ANDC
![Page 10: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/10.jpg)
10
Sensors
DC Location
Sensors
DC Location
App
App
ANDC
Challenges of Data Collection
① Storing sensor data to a far distant server
Long Distant
![Page 11: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/11.jpg)
11
Sensors
DC Location
App
App
ANDC
Challenges of Data Collection
② Single Point of Failure
![Page 12: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/12.jpg)
1212
Sensors
DC Location
App
App
ANDC
Challenges of Data Collection
③ Traffic Congestion
![Page 13: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/13.jpg)
13
Our Proposed Design
![Page 14: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/14.jpg)
Copying Data to the Core Storage
14
![Page 15: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/15.jpg)
15
Nature of Sensor Data
SensorData
Key
Time
Value
1
2
3
Usually sensor data is stored in time series format.
(Key, Time, Value)
![Page 16: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/16.jpg)
16
Data Accessing Mechanism
1
![Page 17: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/17.jpg)
17
Data Accessing Mechanism
2
1
2
1
![Page 18: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/18.jpg)
18
Data Accessing Mechanism
2
1
2
1
Data Verification Respond
3
![Page 19: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/19.jpg)
19
Data Accessing Mechanism
2
1
2
1
Data Verification Respond
3
Send Data4
![Page 20: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/20.jpg)
20
Advantage of the Proposed System
1
Data Availability ① Link disconnection
between Core and Edge Node
![Page 21: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/21.jpg)
21
Advantage of the Proposed System
1
Data Availability ① Link disconnection
between Core and Edge Node
② Core Storage failure
2
![Page 22: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/22.jpg)
22
Advantage of the Proposed System
1
Data Availability ① Link disconnection
between Core and Edge Node
② Core Storage failure
2
![Page 23: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/23.jpg)
23
Our Contributions
Transferring sensor data to the far-distant Core Node
1
Core Storage
Edge Storage
SensorNetwork
![Page 24: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/24.jpg)
24
Our Contributions
Developers design applications by accessing sensor data
2
Core Storage
Edge Storage
SensorNetwork
![Page 25: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/25.jpg)
25
Originality
Available Systems
Distributed Database System • Mostly in one location
inside a SAN• Requires Consistency
Our System
Distributed Storage System• Distributed
geographically in far-distance places
• No consistency because of sensor data
![Page 26: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/26.jpg)
26
Limitation
• The CAP theorem: – Consistency– Availability – Partitioning
Trade-off
Availability
Consistency
![Page 27: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/27.jpg)
27
Challenge
How do clients find or switch to a nearest edge storage?
![Page 28: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/28.jpg)
28
Experiment
InternetCore Storage
JapanAfghanistan
cURL
1
![Page 29: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/29.jpg)
29
Experiment
Internet
Edge Storage
Core Storage
JapanAfghanistan
cURL
cURL
1
2
![Page 30: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/30.jpg)
30
Result
Internet
Edge Storage
Core Storage
JapanAfghanistan
cURL
cURL
8:00 9:00 10:00 11:00 12:000
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
4.3
2.5
3.5
4.5
2.22
1.21
2
0.5
Data from core Storage
Data from Edge Storage
Monday 23rd May 2016
![Page 31: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/31.jpg)
31
Future Work
![Page 32: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/32.jpg)
32
Future Work
P2P Network
![Page 33: A Design of Distributed Storage System over HTTP for Collecting Sensor Data](https://reader035.vdocuments.us/reader035/viewer/2022070520/58f14cbe1a28ab086c8b4629/html5/thumbnails/33.jpg)
33
Thank you“Things should be made as simple as possible, but no simpler.”
Albert Einstein