how big data and cloud computing aspects can be utilized · pdf filehow big data and cloud...

31
ITUTRCSL Symposium on Cloud Computing Colombo, SRI LANKA 2830 July 2015 ITUTRCSL Symposium on Cloud Computing Colombo, SRI LANKA 2830 July 2015 Gus O’Brien Principal Architect Global Solutions Elite Team How Big Data and Cloud Computing aspects can be utilized How Big Data and Cloud Computing aspects can be utilized

Upload: lamcong

Post on 28-Mar-2018

216 views

Category:

Documents


2 download

TRANSCRIPT

ITU‐TRCSL Symposium on Cloud ComputingColombo, SRI LANKA 28‐30 July 2015

ITU‐TRCSL Symposium on Cloud ComputingColombo, SRI LANKA 28‐30 July 2015

Gus O’BrienPrincipal ArchitectGlobal Solutions Elite Team

How Big Data and Cloud Computingaspects can be utilizedHow Big Data and Cloud Computingaspects can be utilized

2

Agenda

Cloud, Big Data and Big Data AnalyticsCloud, Big Data and Big Data Analytics11

22 Developing National Cloud and using Big Data in e‐Learning

Developing National Cloud and using Big Data in e‐Agriculture33

44

55

Considerations in adopting Cloud and Big Data Strategy

Case Studies

3

What is Cloud

• Essential�Characteristics

– On‐demand self‐service– Broadband network access– Resource pooling– Rapid elasticity– Measured 

• Service�Models

– Software as a Service (SaaS).– Platform as a Service (PaaS)– Infrastructure as a Service (IaaS)

• Deployment�Models

– Private cloud– Community cloud– Public cloud– Hybrid cloud

4

Big Data and Big Data Analytics

• Big�Data– Massive�volume�of�both�structured and�unstructured data�that�

is�so�large,�it�is�difficult�to�process�using�traditional database�and�software�techniques.�

– In�most�enterprise�scenarios�the�volume�of�data�is�too�big�or�it�moves�too�fast�or�it�exceeds�current�processing�capacity.�

• Big�Data�Analytics– Process�of�collecting,�organizing�and�analysing�large�sets�of�

data�(called Big�Data)

– Discover�patterns�and�other�useful�information.�

– Help�organizations�to�better�understand�the�information�contained�within�the�data

– Identify�the�data�that�is�most�important�to�the�business�and�future�business�decisions.�

– Big�data�analysts�basically�want�the knowledge that�comes�from�analysing�the�data.

5

Public Cloud and Big Data Trends

• Wikibon,�predict�Big�Data�revenues�of $53.4B�by�2017,�as�more�businesses�begin�to�realize�real�benefits�from�Big�Data�analytics.

6

Developing National Cloud and using Big Data in e-Learning

7

E-Learning Trends

Source:�GSV�ADVISORS

• K-12�(kindergarten�through�12th�grade)�is�the�term�commonly�used�for�talking�about�a�persons�first�13�years�of�schooling�before�entering�university.

• Global�Education�expenditure�has�grown�consistently�in�recent�years�and�the�K-12�sector�plays�a�big�role�in�this�growth

• K-12�sector�50%�CAGR�2012-2017

8

Traditional Learning Challenges• Requires�Physical�location• Teacher�Centred• Limited�number�of�students�per�class• Static�Content• Books• No�24�x�7�access• Does�not�support�distance�based�remote�learning• Non�standard�learning• Not�customisable• Not�cost�effective• Travel�costs• Not�self�paced.

9

How e-learning benefits from cloud computing• E-learning�cloud�can�serve�as�a�reservoir�of�information�

– Students�can�access�learning�when�they�are�connected�through�mobile�devices�or�low�cost�power�tablets– An�educational�cloud�can�leverage�the�power�of�thousands�of�researchers�and�academics�to�focus�on�one�problem

• Lower�cost– Lower�maintenance�costs– No�local�Infrastructure�or�data�storage�costs.

• Automatic�Updated�software– Software�always�kept�up�to�date– All�clients�can�be�updated�at�once.

• Improved�performance– No�reliance�on�local�inadequate�infrastructure

• Asynchronous�learning– Students�can�access�courses��anytime�or�anywhere

• Instructor�Benefits– Courses��and�content�can�be�managed�centrally

• Better�file�compatibility– Risk�of�file�distortion�eliminated

• Improved�security– Less�risk�to�confidential�Data– Less�exposure�to�security�breaches

10

• Data��created�by�students�while�they�are�taking�an�eLearning�course�or�training�module.

• If�a�student�is�interacting�with�a�training�module,�his/her�progress,�assessment�results,�social�sharing,�and�any�other�data�being�produced�during�the�eLearning�course�is�“big�data”.�

• The�Learning Management System,�the�eLearning�Authoring�Tool,�social�media,�multimedia,�etc,�that�have�been�set�by�the�organization�or�the�eLearning�professionals,�collect�the�data.

• The�term�“big�data”�doesn't�only�apply�to�the�volume�of�data�itself,�but�the�individual�pieces�of�data�that�are�being�collected.

• These�pieces�of�data�can�be�analysed�to�offer�schools�or�eLearning�professionals�the�opportunity�to�determine�how�the�learner�is�acquiring�information

• Identify�any�problems�that�may�exist�within�the�eLearning�strategy�itself.

Big Data in terms of the eLearning industry

11

Benefits That Big Data Offer To eLearning • Allow�teachers�to�customize�the�eLearning�experience�

– Provide�learners�with�more�effective,�engaging,�and�informative�eLearning�courses�and�modules.

• Allocate�resources�among�schools�in�a�more�intelligent�manner

• Offering�invaluable�feedback.– Identify�where�the�student,�eLearning�course��be�improved.�– Online�facilitator��can�make�adjustments�to�improve�learners’�performance.

• Allowing�eLearning�professionals�to�design�more�personalized�eLearning�courses.– Modules�can�be�custom�tailored�to�meet�the�individual�needs�of�the�learner– Offer�a�high�quality�and�meaningful�learning�experience.

• Targeting�effective�eLearning�strategies�and�eLearning�goals.– Identify�which�eLearning�strategies�are�working�and�which�ones�are�not– Devote�resources�to�the�aspects�that�are�working,�so�student�receives�the�preparation�they�need�to�fulfil�their�learning�goals.

• Tracking�learner�patterns.– Track�a�learner�throughout�the�entire�process,�from�start�to�finish.�– Pinpoint�patterns�about�the�learning�behaviours�of�the�individual�learner,�and�also�group�as�a�whole.

• Expand�understanding�of�the�eLearning�process.– Better�understanding�of�eLearning�process�and�how�learners�are�responding�to�the�eLearning�courses�

12

Developing National Cloud and using Big Data in e-Agriculture

13

What is e-Agriculture?• An�emerging�field�focusing�on�the�enhancement�of�agricultural�

and�rural�development�through�improved�information�and�communication�processes.�More�specifically,�e-Agriculture�involves�the�conceptualization,�design,�development,�evaluation�and�application�of�innovative�ways�to�use�information�and�communication�technologies�(ICT)�in�the�rural�domain,�with�a�primary�focus�on�agriculture.

14

Why is use of Cloud and Big Data important ?

The�world's�population�will�rise�to�9.1�billion�by�2050�with�food�demand�surging�70�percent�beyond�today's�levels- UN�Food�and�Agricultural�Organization.

15

Traditional Agriculture Challenges• Farms

– Small�traditional�rural�family�farms�run�by�1�or�maybe�2�people– Fragmentation– Poor�Infrastructure– Information�is�shared�verbally�and�mostly�only�amongst�family�members– Agricultural�production�is�decentralized

• Outdated�methods�due�to�lack�of�knowledge– Equipment,�Tools– No�way�of�measuring�crop�yield�– Matching�of�crop�to�soil�type– Crop�rotation

• No�Access�to�Information– Seeds,�Produce�and�fertilisers– Crop�and�livestock�,disease�and�prevention– Weather�Patterns– Inability�to�connect�with�agricultural�experts�to�gain�critical�information

• Supply�Chain– Supply�and�Demand– Too�many� intermediaries�reducing�margins– Constant�cycle�of�low�productivity�and�investment�leads�to�Global�

uncompetitiveness

16

How cloud and Big Data can assist farmers• Cloud�offers�central�knowledge�vault�storing�all�

agricultural�information.• Improved�networking�and�communication• Facilitates�online�trading�and�e-commerce• Farmers��able�to�access�information�through�

kiosks�in�community�centres�on�the�following:

– Ordering supplies and offers (seeds, fertiliser, pesticides etc)– Yield�Information– Crop�Prices– Timely�information�on�weather�forecasts�and�calamities– Better�and�spontaneous�agricultural�practices– Better�marketing�exposure�and�pricing– Access�to�Global�agricultural�experts– Reduction�of�agricultural�risks�and�enhanced�incomes

17

How cloud and Big Data can assist farmers• Big�Data�Analytics

– Seed�Planting• Choice�of�hybrids

– Supply�chain�construction�and�improvement– Analysing�and�sharing�of�weather�forecasts�to�find�the�

right�time�to�plant�and�harvest�crops.– Real-time�tracking,�monitoring�and�assistance�in�

agricultural�production�and�product�quality– Regulator�easily�trace�sources�of�food�when�there�are�

safety�concerns�

18

Agriculture Value Chain Benefits of Cloud and Big Data• Cloud��creates�virtual�cluster�of�all�value�chain�participants,�delivering�the�same�benefits�as�

vertical�integration�does�in�mature�agricultural�economies�like�the�USA�and�Europe

• Unlocking��farmer�potential

– Enhanced competitiveness in agriculture – Trigger cycle of higher productivity– Higher incomes– Greater  investment– Higher quality and productivity.

• Growth�in�rural�incomes�

– Increased demand for industrial goods– Fuels continued growth of local economy. – Propels economy into a higher growth trajectory.

19

IoT in e-Agriculture• The�use�of�big�data�and�the�evolution�of�the�Internet�of�Things�compensates�the�shortcomings�of�traditional�agriculture

• Setting�up�sensors�with�cameras�in�farms��to�collect�data�about�crops,�temperatures,�precipitation�and�humidity.�

• Supplying�Farmers�with�mobile�devices�in�order�to�record�data�relating�to�their�agriculture�processes.– The�data,�along�with�that�recorded�by�farmers�through�mobile�devices�could�then�be�uploaded�to�the�cloud�and�analysed�to�help better�

manage�the�farms

• Farmer�using�a�smart�phone�to�scan�the�barcode�of�a�packet�of�certified�seeds– Something�that�could�ensure�quality�and�a�fair�price– installation�of�low-cost�sensors�at�the�village-level�for�hyper�local�weather�information

• Machine�Data– Aggregating�performance�data�across�a�field�inventory�of�working�machines�has�a�tremendous�benefit�for�companies�and�can�

help�improve�future�designs.

20

Considerations in adopting Cloud and Big Data Strategy

21

Considerations when adopting Cloud• Early�Planning

– Engage�key�stakeholders��from�every��aspect�of�the�business�early�in�process– IT,�Legal,�Security,�Finance�etc

• Understand�Cloud�Models– IasS,�SaaS,�PaaS,�– Different��approaches,�to�securing,�managing,�pricing�and�procuring

• Utility�Pricing�Model– Pay-as-you-go– Pay�only�for�Services�that��are�consumed– Flexibility

• Security– Industry��best��practices,�standards,�auditing�and�controls– Shared�Responsibility�between�provider�and�customer

• Cloud�Data�Governance– Control�and�ownership�of��customer�data.

• Consider�OpenSource�Options– OpenStack�is�now�the�defacto�Cloud�OS– Integration�to�third�parties– Open�API’s– Cloud�deployments�built�around�opensource�are�more�attractive�as�they�prevent�lock-in�and�leverage�global�communities�of�information�

and�development

22

Considerations when selecting Big Data Platform• Data�platform�drivers�

– Storage�or�advanced�analytics?– Storing�and�processing�tens�of�terabytes�of�data,�using�an�open-source�distributed�file�system�is�a�mature�choice�due�to�its�

predictable�scalability�over�clustered�hardware.�– run�analytics�in�online�or�real-time�applications,�hybrid�architectures�containing�distributed�file�systems�combined�with�

distributed�database�management�systems�(which�have�lower�latency).

• Low�latency,�Real�time�application– Consider�In-memory�solution

• Availability�and�Consistency�requirements�for�the�platform– CAP�theorem-- systems�can’t�be�both�consistent�and�available�when�a�break�in�the�network�causes�the�system�to�be�

partitioned.– trading�off�consistency�for�availability.– Modern�systems�have�developed��strategies�for�dealing�with�partitioning�scenarios,�reducing�the�impact�and�improving�the�

ability�of�systems�to�recover�from�partitions�in�a�less�disruptive�way

• How�is�data�be�accessed�by�users�and�applications– Specific�application�interfaces�(APIs)��for�NoSQL�Databases

• Workload�Profile– Consider�spikes�in�in�workload

23

Summary• Leveraging�Cloud�and�Big�Data�can�overcome�traditional�challenges�posed�by�Education�and�Agriculture

– Reservoir�of�Information– No�or�low�cost– Centralised– Real�time,�on-demand�access– Knowledge�management�efficiency– Access�to�global�experts

• E-Learning– Tailor�Content�to�students�needs– Self-Paced�e-Learning– Tracking��student�and�improve�content– Allocate�resources�more�intelligently

• E-Agriculture– Use�of�Big�Data�to�select�seeds,�weather�patterns– Eliminate�intermediaries– Supply�chain– Real-Time�tracking�and�monitoring�to�ensure�produce�quality– Leads��to�increased�income,�productivity�in�farming

• Plan�Cloud�and�Big�Data�strategy�early�and�thoroughly

24

Huawei in Cloud and Big Data

Page 24

25

Huawei Cloud Computing and Big Data Efforts and Recognition

Gartner• Member of the Magic

Quadrant for x86 Server Virtualization Infrastructure

• Member of the Magic Quadrant for Integrated Systems

Top 2New market shares obtained in China in 2014

10,000Huawei employees working in cloud computing and dedicated to meeting every IT requirement

No.1Industry-leading performance according to the SPECvirtserver virtualization performance benchmark

Frost & Sullivan 2013Cloud Infrastructure Product Innovation AwardBBTM Award 2014“Most Innovative Tool for Driving Real-Time Intelligence” award at the Broadband Traffic Management (BBTM) & Telco Big Data Summit

OpenStackGold Member

TPC-BDWGKey Member

100,000Desktops in the world's largest-scale deployment

26

Case Studies

27

School of Electronics Engineering and Computing Science, Peking University is first choice of HANA ISV in China for SAP. The School is leading the industry of governmental security Big Data application.

The governmental security industry requires the SAP HANA to run based on high performance servers, which requires the extremely high reliability and computing performance of the hardware platform.

Challenges

Huawei RH5885 V2 rack servers are used to build the SAP HANA appliance. RH5885 V2 server is the first server in China that has been certified by SAP HANA, enjoys 35 RAS features, supports Intel E7 series processors. Thus, the servers are highly reliable and of high performance.

The RH5885 V2 is configured with dual 10GE NICs and multiple GE NICs to expand the network bandwidth, Huawei ES3000 PCIe SSD card to improve the storage I/O performance and to ensure the high-speed service running.

Solutions

The hardware application requirements of the governmental security industry are fulfilled by the high reliability and performance compatibility of the SAP HANA appliance that runs on high performance servers.

Huawei innovative SSD card improves the appliance I/O performance by 5 times, saves the hardware investments by 60%, and hence enhances the advantages of the appliance.

Customer Benefits

Huawei and College of Information Science and Technology of Peking University - Big

Data Research

28

Tsinghua University's Institute for Interdisciplinary Information Sciences wanted to build a world-class interdisciplinary information research center and talent-fostering base. The institute's Hadoop big data research platform required high computing and I/O performance of numerous servers.

The institute has not been established long and has no professional equipment rooms. It is a big challenge to build a modern Big Data research platform in old buildings.

Challenges

"We are impressed by Huawei's advanced technologies and creative engineers. Huawei servers are built on a well-designed architecture. They offer optimal performance and adopt unique design in structure and cooling to adapt to our teaching environment. As a result, the big data research platform can easily address our research requirements."

Tsinghua University Builds a Hadoop Big Data Research Platform with Huawei Servers

To meet performance requirements of the research platform, Huawei provided RH2288 V2 servers. Each RH2288 V2 server supports the latest Intel processors, twelve 3 TB and two 1 TB hard disks, and Huawei's 10GE NICs to provide high computing performance, large storage capacity, and outstanding system I/O performance.

To address the customer's difficulties in equipment room design, Huawei provided plans for reforming the distributed data center, weak and strong current, and fire extinguishing system.

Solution

The Hadoop Big Data research platform built on RH2288 V2 servers has super performance.

Huawei's solution enables distributed deployment in existing buildings, unified management, and simplified O&M, which helps slash construction and O&M costs by 40%.

Customer Benefits

---- Tsinghua University's Institute for Interdisciplinary Information Sciences

29

Guangzhou Education Cloud: Sharing of Education Resources Leads to the Balanced Development of Education

The Education Bureaus of Guangzhou are composed of Education Bureaus of 10 districts and two counties, but the education development in each district and county is imbalanced. The Education Bureau of Guangzhou plans to eliminate the digital divide of education within three years and realize the equalization of education between city and countryside in Guangzhou. Although the education of Guangzhou is top-ranked nationwide, several suburban districts and counties of Guangzhou such as Conghua, Zengcheng, and Baiyun District still fall behind in informatization, and the excellent experience of many schools of Guangzhou is difficult to be shared with schools that fall behind. More than 60,000 PCs are phased out by schools of districts and counties every year, causing a high purchasing cost.

The Education Bureaus of Guangzhou are composed of Education Bureaus of 10 districts and two counties, but the education development in each district and county is imbalanced.

The imbalance of education is eliminated, and special attention is paid to the high quality teaching resources.

The utilization rate of the education resources is increased, and a waste of resources is reduced.

Challenges

Equalization of education resources is realized through the remote education and resources sharing.

Resource capability is allocated dynamically through the elastic computing of cloud computing, and then the utilization rate of resources is greatly increased.

The digital education content and diversified access terminals allow the education resources to be visited at any time and anywhere.

Huawei Solution

A cloud platform of education resources is built, and equalization of education resources is realized through the remote education and resources sharing.

The cloud computing greatly increases the utilization rate of resources, and allows the education resources to be visited at any time and anywhere.

IT system investment is reduced, and the life cycle of IT devices is extended.

Customer Benefits

30

A variety of government services and low resource utilization cause a serious waste of information resources and O&M investments.

The existing IT facilities lag behind the application development, which adversely affects the e-Government service level. An efficient DC is required urgently to optimize the ROI.

Challenges

Laos national DC

Provides end-to-end DC solutions from Layer 1 to Layer 4 and uses a container DC as the DR center.

• Cloud DC: develops an efficient, reliable, and energy-efficient cloud DC and unified management platform.

• Rich cloud services: provides various cloud services covering commerce, Agriculture, and finance for Laotian government, enterprises, and citizens.

• Equipment room facilities: uses the energy-efficient equipment room IDS2000\1000 to support flexible expansion and energy savings.

• Professional services: provides Huawei proprietary services such as technical trainings, service consultation, and optimization.

Solution

•The government information and work process are open, achieving transparent government affairs.

•The information sharing and construction of department informatization improve the work efficiency of employees.

•The public can learn about the basic information about Laotian government easily.

•The public involvement is enhanced to allow more people to participate in the discussions on government affairs.

Customer benefits

Copyright©2015 Huawei Technologies Co., Ltd. All Rights Reserved.The information in this document may contain predictive statements including, without limitation, statementsregarding the future financial and operating results, future product portfolio, new technology, etc. There are anumber of factors that could cause actual results and developments to differ materially from those expressed orimplied in the predictive statements. Therefore, such information is provided for reference purpose only andconstitutes neither an offer nor an acceptance. Huawei may change the information at any time without notice.