cloud computing - summary · 2016-03-29 · internet of things (iot) • “the internet of things...

Basics of Cloud Computing – Lecture 8

Mobile Cloud, Internet of Things and

Cloud Computing – Summary and

Preparation for Examination

Satish Srirama

Outline

• Continue with Research at Mobile & Cloud Lab

• Quick recap of what we have learnt as part of

this course

• How to prepare for the examination

3/29/2016 Satish Srirama 2/38

3/29/2016

[Tomi T Ahonen]

Satish Srirama 3/38

Mobile Applications

• One can do interesting things on mobiles directly

– Today’s mobiles are far more capable

– Location-based services (LBSs), mobile social networking, mobile commerce, context-aware services etc.

• It is also possible to make the mobile a service provider

– Mobile web service provisioning [Srirama et al, ICIW 2006; Srirama

and Paniagua, MS 2013]

– Challenges in security, scalability, discovery and middleware are studied [Srirama, PhD 2008]

– Mobile Social Network in Proximity [Chang et al, ICSOC 2012; PMC

2014]


However, we still have not achieved

• Longer battery life

– Battery lasts only for 1-2 hours for continuous computing

• Same quality of experience as on desktops

– Weaker CPU and memory

– Storage capacity

• Still it is a good idea to take the support of external resources for building resource intensive mobile applications


Mobile Cloud Applications

• Bring the cloud infrastructure to the proximity

of the mobile user

• Mobile has significant advantage by going

cloud-aware

– Increased data storage capacity

– Availability of unlimited processing power

– PC-like functionality for mobile applications

– Extended battery life (energy efficiency)


Mobile Cloud – Our interpretation

• We do not see Mobile Cloud to be just a scenario where mobile is taking the help of a much powerful machine!!!

• We do not see cloud as just a pool of virtual machines

• Mobile Cloud based system should take advantage of some of the key intrinsic characteristics of cloud efficiently

– Elasticity & AutoScaling

– Utility computing models

– Parallelization (e.g., using MapReduce)


Mobile Cloud Binding Models

3/29/2016

Task Delegation Code Offloading

[Flores & Srirama, JSS 2014]

[Flores et al, MoMM 2011]

Mobile Cloud

[Flores and Srirama, MCS 2013]

Satish Srirama 8/38

MCM – enables

• Interoperability between different Cloud Services (IaaS, SaaS, PaaS) and Providers (Amazon, Eucalyptus, etc)

• Provides an abstraction layer on top of API

• Composition of different Cloud Services

• Asynchronous communication between the device and MCM [Warren et al, IEEE PC 2014]

• Means to parallelize the tasks and take advantage of Cloud’s intrinsic characteristics

3/29/2016

[Flores et al, MoMM 2011]

Satish Srirama 9/38

MCM applications

• CroudSTag [Srirama et al, MobiWIS 2011]

– Social group formation with people identified in Pictures/Videos

• Zompopo [Srirama et al, NGMAST 2011]

– Intelligent calendar, by mining accelerometer sensor data

• Bakabs [Paniagua et al, iiWAS-2011]

– Managing the Cloud resources from mobile

• Sensor data analysis– Human activity recognition

– Context aware gaming

– MapReduce based sensor data analysis [Paniagua et al, MobiWIS 2012]

• SPiCa: A Social Private Cloud Computing Application Framework [Chang et al, MUM 2014]


Code Offloading - Major Components

• Major research challenges – What, when,

where and how to offload?

• Mobile– Code profiler

– System profilers

– Decision engine

• Cloud based surrogate platform

3/29/2016

[Flores and Srirama, MCS 2013]

Satish Srirama 11/38

Challenges and technical problems

• Inaccurate code profiling– Code has non-deterministic behaviour during runtime

• Based on factors such as input, type of device, execution environment, CPU, memory etc.

– Some code cannot be profiled (e.g. REST)

• Integration complexity– Dynamic behaviour vs Static annotations

• E.g. Static annotations cause unnecessary offloading

• Dynamic configuration of the system

• Offloading scalability and offloading as a service– Surrogate should have similar execution environment

– Should also consider about resource availability of Cloud

3/29/2016

[Flores et al, IEEE Communications Mag 2015]

Satish Srirama 12/38

Practical adaptability of offloading

Applications that can benefit became limited with increase in device capacities3/29/2016 Satish Srirama 13/38

Internet of Things (IoT)

• “The Internet of Things allows people and

things to be connected Anytime, Anyplace,

with Anything and Anyone, ideally using Any

path/network and Any service.” [European

Research Cluster on IoT]

• More connected devices than people

• Cisco believes the market size will be $19

trillion by 2025

3/29/2016 Satish Srirama14/38

IoT - Scenarios

• Environment Protection

• Smart Home

• Smart Healthcare

• Smart Agriculture

[Kip Compton][Perera et al, TETT 2014]


Internet of Things – Challenges

Sensors Tags Mobile Things

Appliances & Facilities

How to interact

with ‘things’

directly?

How to provide

energy efficient

services?

How do we

communicate

automatically?

[Chang et al, ICWS 2015]

[Chang et al, SCC 2015;

Liyanage et al, MS 2015]


Cloud-based IoT

Sensing and smart devices

Connectivity nodes &

Embedded processing

Remote Cloud-based

processing

Proxy Storage

Processing


Further research focus in IoT

• We have established IoT and Smart Solutions

Lab with Telia company support

• Interesting topics

– Discovery of IoT devices

– Study of IoT based middlewares

– Study of available IoT platforms

• Amazon IoT

• Open IoT


IoT Data Processing on Cloud

• Enormous amounts of unstructured data– In Zetabytes (1021 bytes) by 2020 [TelecomEngine]

– Has to be properly stored, analysed and interpreted and presented

• Big data acquisition and analytics– Is MapReduce sufficient?

• MapReduce is not good for iterative algorithms [Srirama et al, FGCS 2012]

– IoT mostly deals with streaming data• Message queues such as Apache Kafka can be used to buffer and feed

the data into stream processing systems such as Apache Storm

• Apache Spark streaming

• How to ensure QoS aspects such as security of data?– Anonymization and Expiry of data?

• Especially for the personal data


WE ALWAYS WELCOME NEW IDEAS!

email: [email protected]

3/29/2016 Satish Srirama 20

WHAT WE LEARNT IN THE COURSE!

3/29/2016 Satish Srirama 21

What is Cloud Computing?

• Computing as a utility

– Consumers pay based on their usage

• Cloud Computing characteristics

– Illusion of infinite resources

– No up-front cost

– Fine-grained billing (e.g. hourly)

• Gartner: “Cloud computing is a style of computing where massively scalable IT-related capabilities are provided ‘as a service’ across the Internet to multiple external customers”


Cloud Computing - Services• Software as a Service – SaaS

– A way to access applications hosted on the web through your web browser

• Platform as a Service – PaaS– Provides a computing platform

and a solution stack (e.g. LAMP) as a service

• Infrastructure as a Service –IaaS– Use of commodity computers,

distributed across Internet, to perform parallel processing, distributed storage, indexing and mining of data

– Virtualization

SaaS

Facebook, Flikr, Myspace.com,

Google maps API, Gmail

PaaS

Google App Engine, Force.com, Hadoop, Azure,

Heroku, etc

IaaS

Amazon EC2, Rackspace, GoGrid, SciCloud, etc.

Level of

Abstraction


Cloud Computing - Themes

• Massively scalable

• On-demand & dynamic

• Only use what you need - Elastic – No upfront commitments, use on short term basis

• Accessible via Internet, location independent

• Transparent – Complexity concealed from users, virtualized,

abstracted

• Service oriented – Easy to use Service Level Agreements


Cloud Computing Progress

• Short term and long term implications of cloud

• Economics of cloud users and cloud providers

• Challenges and opportunities offered by cloud

computing


Cloud Providers

• Amazon Web Services– EC2, S3, EBS, Elastic Load Balancing, Amazon Auto Scale,

Amazon CloudWatch, IAM, Elastic Beanstalk, CloudFormation, Simple Workflow Service, Elastic MapReduce

• Private Cloud enabling technologies– Eucalyptus

– OpenStack• Worked with SciCloud

• PaaS– Google AppEngine

– Windows Azure


Scaling Applications on the Cloud

• Two basic models of scaling

– Vertical scaling, aka Scale-up

– Horizontal scaling, aka Scale-out

• Scaling Enterprise Applications in the Cloud

• Load balancing

– Types and algorithms

• Autoscaling


Economics of Cloud Providers

• Cloud Computing providers bring a shift from high reliability/availability servers to commodity servers– At least one failure per day in large datacenter

• Why?– Significant economic incentives

• much lower per-server cost

• Caveat: User software has to adapt to failures– Very hard problem!

• Solution: Replicate data and computation– MapReduce & Distributed File System


MapReduce

• Programmers specify two functions:map (k, v) → <k’, v’>*reduce (k’, v’) → <k’, v’>*– All values with the same key are reduced together

• The execution framework handles everything else…• Not quite…usually, programmers also specify:

partition (k’, number of parWWons) → parWWon for k’– Often a simple hash of the key, e.g., hash(k’) mod n– Divides up key space for parallel reduce operationscombine (k’, v’) → <k’, v’>*– Mini-reducers that run in memory after the map phase– Used as an optimization to reduce network traffic


Hadoop Processing Model

• Create or allocate a cluster

• Put data onto the file system (HDFS)– Data is split into blocks

– Replicated and stored in the cluster

• Run your job– Copy Map code to the allocated nodes

• Move computation to data, not data to computation

– Gather output of Map, sort and partition on key

– Run Reduce tasks

• Results are stored in the HDFS


MapReduce Examples

• Distributed Grep

• Count of URL Access Frequency

• Reverse Web-Link Graph

• Inverted Index

• Distributed Sort


Synchronization in Hadoop

• Approach 1: turn synchronization into an ordering problem– Sort keys into correct order of computation

– Partition key space so that each reducer gets the appropriate set of partial results

– Hold state in reducer across multiple key-value pairs to perform computation

– Illustrated by the “pairs” approach in calculating conditional probability of words

• Approach 2: construct data structures that “bring the pieces together”– Each reducer receives all the data it needs to complete the

computation

– Illustrated by the “stripes” approach


Research @ Mobile & Cloud Lab

• Scientific computing on the cloud

• Classification on how the algorithms can be

adapted to MR

• Limitations of Hadoop MapReduce

– Alternatives

• Mobile Cloud

– Mobile Cloud Binding Models

• Cloud based Internet of Things


This week in lab

• Complete the work from previous labs


Examination

• Exam I : Monday, 11.04.2016, 10:00 – 12:00

Room yet to be decided

• Exam II : Tuesday, 12.04.2016, 10:00 – 12:00 in

room J. Liivi 2 - 402


Preparation for Examination

• One of the earlier exam paper is kept online

• Slides are mostly self sufficient

• References are mentioned for further reading

• Mainly focus at keywords

• Let us discuss some questions…


References

• Basics of Cloud Computing – Lectures https://courses.cs.ut.ee/2016/cloud/spring/Main/Lectures

• List of Publications - Satish Narayana Srirama - http://math.ut.ee/~srirama/publications.html

• [Flores et al, IEEE Communications Mag 2015] H. Flores, P. Hui, S. Tarkoma, Y. Li, S. N. Srirama, R. Buyya: Mobile Code Offloading: From Concept to Practice and Beyond, IEEE Communications Magazine, ISSN: 0163-6804, 53(3):80-88, 2015. IEEE. DOI:10.1109/MCOM.2015.7060486

• [Chang et al, SCC 2015]C. Chang, S. N. Srirama, J. Mass: A Middleware for Discovering Proximity-based Service-Oriented Industrial Internet of Things, 12th IEEE International Conference on Services Computing (SCC 2015), June 27 - July 2, 2015, pp. 130-137. IEEE.

• [Liyanage et al, MS 2015] M. Liyanage, C. Chang, S. N. Srirama: Lightweight Mobile Web Service Provisioning for Sensor Mediation, 4th International Conference on Mobile Services (MS 2015), June 27 - July 2, 2015, pp. 57-64. IEEE

• [Chang et al, ICWS 2015] C. Chang, S. Loke, H. Dong, F. Salim, S. N. Srirama, M. Liyanage, S. Ling: An Energy-Efficient Inter-organizational Wireless Sensor Data Collection Framework, The IEEE 22nd International Conference on Web Services (ICWS 2015), June 27 - July 2, 2015, pp. 639-646. IEEE.

• [Flores and Srirama, JSS 2014] H. Flores, S. N. Srirama: Mobile Cloud Middleware, Journal of Systems and Software, ISSN: 0164-1212, 92(1):82-94, 2014. Elsevier. DOI: 10.1016/j.jss.2013.09.012.

• [Chang et al, PMC 2014] C. Chang, S. N. Srirama, S. Ling: Towards an Adaptive Mediation Framework for Mobile Social Network in Proximity, Pervasive and Mobile Computing Journal, MUCS Fast track, ISSN: 1574-1192, 12:179-196, 2014. Elsevier. DOI: 10.1016/j.pmcj.2013.02.004.

• [Warren et al, IEEE PC 2014] I. Warren, A. Meads, S. N. Srirama, T. Weerasinghe, C. Paniagua: Push Notification Mechanisms for Pervasive Smartphone Applications, IEEE Pervasive Computing, ISSN: 1536-1268, 13(2):61-71, 2014. IEEE. DOI:10.1109/MPRV.2014.34.

• [Chang et al, MUM 2014] C. Chang, S. N. Srirama, S. Ling: SPiCa: A Social Private Cloud Computing Application Framework, The 13th International Conference on Mobile and Ubiquitous Multimedia (MUM 2014), November 25-28, 2014, pp. 30-39. ACM.

• [Jakovits and Srirama, HPCS 2014] P. Jakovits, S. N. Srirama: Evaluating MapReduce Frameworks for Iterative Scientific Computing Applications, The 2014 (12th) International Conference on High Performance Computing & Simulation (HPCS 2014), July 21-25, 2014, pp. 226-233. IEEE.

• [Kromonov et al, HPCS 2014] I. Kromonov, P. Jakovits, S. N. Srirama: NEWT - A resilient BSP framework for iterative algorithms on Hadoop YARN, The 2014 (12th) International Conference on High Performance Computing & Simulation (HPCS 2014), July 21-25, 2014, pp. 251-259. IEEE.

• [Srirama and Viil, HPCC 2014] S. N. Srirama, J. Viil: Migrating Scientific Workflows to the Cloud: Through Graph-partitioning, Scheduling and Peer-to-Peer Data Sharing, 16th Int. Conf. on High Performance and Communications (HPCC 2014) workshops, August 20-22, 2014, pp. 1137-1144. IEEE.

• [Jakovits and Srirama, Nordicloud 2013] P. Jakovits, S. N. Srirama: Clustering on the Cloud: Reducing CLARA to MapReduce, 2nd Nordic Symposium on Cloud Computing & Internet Technologies (NordiCloud 2013), September 02-03, 2013, pp. 64-71. ACM.

• [Jakovits et al, HPCS 2013] P. Jakovits, S. N. Srirama, I. Kromonov: Viability of the Bulk Synchronous Parallel Model for Science on Cloud, The 2013 (11th) International Conference on High Performance Computing & Simulation (HPCS 2013), July 01-05, 2013, pp. 41-48. IEEE.

• [Srirama and Paniagua, MS 2013] S. N. Srirama, C. Paniagua: Mobile Web Service Provisioning and Discovery in Android Days, The 2013 IEEE International Conference on Mobile Services (MS 2013), June 27 - July 02, 2013, pp. 15-22. IEEE.

• [Flores and Srirama, MCS 2013] H. Flores, S. N. Srirama: Adaptive Code Offloading for Mobile Cloud Applications: Exploiting Fuzzy Sets and Evidence-based Learning, The Fourth ACM Workshop on Mobile Cloud Computing and Services (MCS 2013) @ MobiSys 2013, June 25-28, 2013, pp. 9-16. ACM.

• [Oliner et al, 2013] Oliner, Adam J., Anand P. Iyer, Ion Stoica, Eemil Lagerspetz, and Sasu Tarkoma. "Carat: Collaborative energy diagnosis for mobile devices." In Proceedings of the 11th ACM Conference on Embedded Networked Sensor Systems, p. 10. ACM, 2013.

• [Srirama et al, SOCA 2012] S. N. Srirama, C. Paniagua, H. Flores: Social Group Formation with Mobile Cloud Services, Service Oriented Computing and Applications Journal, ISSN: 1863-2386, 6(4):351-362, 2012. Springer. DOI: 10.1007/s11761-012-0111-5.

• [Srirama et al, FGCS 2012] S. N. Srirama, P. Jakovits, E. Vainikko: Adapting Scientific Computing Problems to Clouds using MapReduce, Future Generation Computer Systems Journal, 28(1):184-192, 2012. Elsevier press. DOI 10.1016/j.future.2011.05.025.


References - continued• [Chang et al, ICSOC 2012] C. Chang, S. N. Srirama, S. Ling: An Adaptive Mediation Framework for Mobile P2P Social

Content Sharing, 10th International Conference on Service Oriented Computing (ICSOC 2012), November 12-16, 2012, pp. 374-388. Springer LNCS.

• [Paniagua et al, MobiWIS 2012] C. Paniagua, H. Flores, S. N. Srirama: Mobile Sensor Data Classification for Human Activity Recognition using MapReduce on Cloud, The 9th Int. Conf. on Mobile Web Information Systems (MobiWIS2012), August 27-29, 2012, pp. 585-592. Elsevier.

• [Srirama et al, SPJ 2011] S. N. Srirama, O. Batrashev, P. Jakovits, E. Vainikko: Scalability of Parallel Scientific Applications on the Cloud, Scientific Programming Journal, Special Issue on Science-driven Cloud Computing, 19(2-3):91-105, 2011. IOS Press. DOI 10.3233/SPR-2011-0320.

• [Flores et al, MoMM 2011] H. Flores, S. N. Srirama, C. Paniagua: A Generic Middleware Framework for Handling Process Intensive Hybrid Cloud Services from Mobiles, The 9th International Conference on Advances in Mobile Computing & Multimedia (MoMM-2011), December 5-7, 2011, pp. 87-95. ACM.

• [Paniagua et al, iiWAS 2011] C. Paniagua, S. N. Srirama, H. Flores: Bakabs: Managing Load of Cloud-based Web Applications from Mobiles, The 13th International Conference on Information Integration and Web-based Applications & Services (iiWAS-2011), December 5-7, 2011, pp. 489-495. ACM.

• [Srirama et al, MobiWIS 2011] S. N. Srirama, C. Paniagua, H. Flores: CroudSTag: Social Group Formation with Facial Recognition and Mobile Cloud Services, The 8th International Conference on Mobile Web Information Systems (MobiWIS 2011), September 19-21, 2011, v. 5 of Procedia Computer Science, pp. 633-640. Elsevier. doi: 10.1016/j.procs.2011.07.082.

• [Srirama et al, NGMAST 2011] S. N. Srirama, H. Flores, C. Paniagua: Zompopo: Mobile Calendar Prediction based on Human Activities Recognition using the Accelerometer and Cloud Services, 5th International Conference on Next Generation Mobile Applications, Services and Technologies (NGMAST 2011), September 14-16, 2011, pp. 63-69. IEEE.

• [Batrashev et al, HPCS 2011] O. Batrashev, S. N. Srirama, E. Vainikko: Benchmarking DOUG on the Cloud, The 2011 International Conference on High Performance Computing & Simulation (HPCS 2011), July 4-8, 2011, pp. 677-685. IEEE.

• [Srirama, PhD 2008] S. N. Srirama: Mobile Hosts in Enterprise Service Integration, PhD thesis, RWTH Aachen University, September, 2008.

• [Srirama et al, ICIW 2006] S. N. Srirama, M. Jarke, W. Prinz: Mobile Web Service Provisioning, Proceedings of the Advanced International Conference on Telecommunications and International Conference on Internet and Web Applications and Services (AICT-ICIW 2006), February 23-25, 2006, pp. 120-125. IEEE Computer Society Press.

• [Valiant, 1990] L. G. Valiant: A bridging model for parallel computation, Commun. ACM, vol. 33, no. 8, pp. 103–111, Aug. 1990.


cloud computing - summary · 2016-03-29 · internet of things (iot) • “the internet of things...

Documents