school ofengineering andnatural sciences, university

PARALLEL & SCALABLE MACHINE LEARNING & DEEP LEARNING

Prof. Dr. – Ing. Morris RiedelAdjunct Associated ProfessorSchool of Engineering and Natural Sciences, University of IcelandResearch Group Leader, Juelich Supercomputing Centre, Germany

Platform-As-A-Service (PAAS)November 6th, 2018Room VO2 – 138

Cloud Computing & Big Data

LECTURE 9

Review of Lecture 8 – Infrastructure-As-A-Service (IAAS)

IAAS Provisioning

Lecture 9 – Platform-As-A-Service (PAAS)

Not only compute resources, but also storage resources…

[1] Distributed & Cloud Computing Book [2] Key-Based Authentication

(e.g. AWS SimpleStorage ServiceS3 is a scalableobject storage)

Choosing Deployments

modified from [3] Accenture Deployment Models

(example: map-reduce that can beconsidered as an infrastructure for some users)

2 / 55

Outline of the Course

1. Cloud Computing & Big Data

2. Machine Learning Models in Clouds

3. Apache Spark for Cloud Applications

4. Virtualization & Data Center Design

5. Map-Reduce Computing Paradigm

6. Deep Learning driven by Big Data

7. Deep Learning Applications in Clouds

8. Infrastructure-As-A-Service (IAAS)

9. Platform-As-A-Service (PAAS)

10. Software-As-A-Service (SAAS)


11. Data Analytics & Cloud Data Mining

12. Docker & Container Management

13. OpenStack Cloud Operating System

14. Online Social Networking & Graphs

15. Data Streaming Tools & Applications

16. Epilogue

+ additional practical lectures for our

hands-on exercises in context

Practical Topics

Theoretical / Conceptual Topics3 / 55


Outline

Understanding PAAS Environments Different Cloud Service Levels Reviewed Google Cloud Products & PAAS Examples Google App Engine (GAE) & NoSQL Services GAE Software Development Kit (SDK) Philipps Application Example & GAE Pricing

Advanced PAAS Topics Startup Voodoo Quiz Run Application Example PAAS Services Big Query & Machine Learning GAE AppStats & MemCache Features Rovio Games Angry Birds Application Example Google Cloud TPU & Machine Learning Services

Promises from previous lecture(s):

Lecture 1: Lecture 8 & 9 & 10 offer more insights into concrete cloud systems and their architectures today

Lecture 3 & 5: Lecture 9 provides more detailedinformation about Google Cloud Platform-As-A-Service Tools

Lecture 4: Lecture 8 & 9 & 10 offer more insights into concrete cloud systems and their use of virtualization

4 / 55

Understanding PAAS Environments

Lecture 9 – Platform-As-A-Service (PAAS) 5 / 55


Three Levels of Cloud Service Models: *AAS

Levels oriented towards different users Full customization to direct usable applications

Software as a Service (SAAS) Provides specific ‘ready-to-run applications’ Sometimes related to geographical location

Platform as a Services (PAAS) Virtual images ready to deploy your software Includes a ‘platform for creation of your services’

Infrastructure as a Service (IAAS) Provides ‘bare metal infrastructure’ & virtual IT resources (cf. Lecture 8) Use and tune infrastructure as needed (compute, storage, networking, …)

Cloud computing infrastructures typically offer services on three different levels: Infrastructure as a Service (IAAS), Platform as a Service (PAAS), and Software as a Service (SAAS)

[4] Wikipedia‘Cloud computing’

focus in this lecture

6 / 55

Different Cloud Service Models – PAAS – Revisited

Platform-As-A-Service (PAAS) E.g. used to provision billing services, handle compute job queing,

launching of images, and monitoring to support application developers


The Conceptual ideas and key usage of the PAAS cloud service model is building cloud applications with software development kits (SDKs) & application programming Interfaces (APIs) via basic services

PAAS is based on known application frameworks similiar to ASP, J2EE, JSPand languages like Python, Java, Ruby, etc.

[1] Distributed & Cloud Computing Book7 / 55


Modified from[1] Distributed & Cloud Computing Book

PAAS – Building Cloud Solutions Overview

[20] Lego Bricks

8 / 55

Google Cloud – Cloud Computing Products & Categories


[19] Google Cloud – Compute Products

9 / 55

Google Cloud – Cloud Computing – Compute & Storage


The Google Cloud Platform offers a wide variety of IAAS/PAAS/SAAS technologies in the areas of compute, storage and databases, networking, big data, machine learning, management tools, developer tools, as well as identity and security – many of them can be combined for applications

The Google App Engine (GAE) PAAS solution is taking advantage of the IAAS Google infrastructure

Lecture 12 provides more details about Kubernetes, containers & registries including tool Docker

(PAAS level)

(IAAS level)

(follows in Lecture 12)


10 / 55

PAAS Example – Google App Engine (GAE)

Platform for building scalable Web applications Based on the Google Cloud Platform – ‘Compute‘ area Offers built-in services common to most applications Upload app code and Google manages app availability No need for servers to buy and/or maintain Scales automatically from 1 to million users Supports known programming languages like Java, Python, PHP or Go

Selected Business Cases Use powerful platform & existing services to manage apps for ‘uncertainty‘ Build server-based Apps that instantly launch as many servers as required


Google App Engine (GAE) is a PAAS-based cloud model for building scalable Web applications andmobile backends that scale automatically in response to the amount of traffic they receive

GAE offers services of the Google infrastructure common to most applications: NoSQL datastore, user authentication API, memcache, Google cloud SQL, Google-like search, load balancing, healthchecks, application logging, security vulnerabilities scans, traffic splitting, task queues, etc.

[5] Google App Engine Web page

11 / 55

GAE based on Google Cloud – Examples


NoSQL Cloud Datastore Provides schemaless object datastore with scalable storage Includes a rich data modeling API & SQL-like query language

User Authentication API Enables authentication via known existing Google accounts

Memcache Offers a distributed in-memory data cache (improved app performance)

Google Cloud SQL Access to a Web service to use relational databases in the cloud (MySQL)

Google-like Search Performs searches over structured data (text, HTML, geo-locations, etc.)

Task Queues Execute work outside of user requests with small tasks performed ‘later‘

[5] Google App Engine Web page

12 / 55

What means NoSQL?

Motivations Simplicity of design, horizontal scaling,

and finer control over availability

‘Implementation concepts’ of NoSQL Databases Often highly optimized ‘key–value stores’ Intended for ‘simple retrieval and appending operations’ (no update!) Increasing ‘latency’ and ‘throughput’

Big Data – ‘Accumulate data over time’ Data is indexed using ‘row’ and ‘column names’ (e.g. arbitrary strings) Data are represented by ‘uninterpreted strings’ (but clients often serialize

various forms of structured and semi-structured data into strings)Lecture 9 – Platform-As-A-Service (PAAS)

NoSQL databases provide mechanisms for storage and retrieval of data that employs less constrained consistency models than traditional relational databases (i.e. non-relational databases)

NoSQL databases are also referred to as ‘Not only SQL’ to indicate that SQL-like queries still usable

[21] Wikipedia on ‘NoSQL’

(e.g. databases optimizedfor reading from data)

[20] Lego Bricks

13 / 55

Philipps Example – GAE & NoSQL Cloud Datastore (1)

Business Case Enhance interactice home lighting with Philipps Hue Control home lighting from smartphone apps Revenue: new product lines interconnected with phones

Benefits Frees engineers to work on product

development rather than managing IT Serve 200 million transactions per day

Approach Use Google Cloud Platform with GAE

and NoSQL-based Google Cloud Datastore


[6] Philipps Case Study

Philipps connects light bulbs to the Internet and offers interactive app by using the GAE GAE provides a powerful backend PAAS solution that let Philipps apps securely access, monitor,

and interact through the Internet connected Philipps Hue Bridge with the home lightning system

14 / 55

Philipps Example – GAE & NoSQL Cloud Datastore (2)

Business Case Enhance interactice home lighting with Philipps Hue Control home lighting from smartphone apps Revenue: new product lines interconnected with phones

Facts & Statements


[6] Philipps Case Study

Benefits for Philipps & other Cloud PAAS users is to free engineers towork on product developmentrather than managing infrastructure

15 / 55

GAE – Using NoSQL-based Cloud Datastore

NoSQL Features Built for automatic scaling and high performance (including replication) Easier application development (e.g. no database schema means

changes to the underlying data structure possible as application evolve) Interface similar like SQL traditional databases (API with REST calls) Different way how relationships are described (compared to SQL) Offers powerful query engine with search for data across properties (+sort)

Philipps application example E.g. store data like which light is

on/off in which room of whichbuilding in which country…

E.g. flexible data structure, lampmoved from room A into room B

E.g. scaling the solution whilegrowing with number of users

Lecture 9 – Platform-As-A-Service (PAAS)[7] Google Cloud Datastore

[20] Lego Bricks

16 / 55

Google Cloud – Cloud Computing – Databases


GAE services re-use other existing services of the Google Cloud Platform through well defined interfaces such as the Cloud SQL service or the NoSQL services Cloud BigTable & Cloud Datastore


[20] Lego Bricks

17 / 55

Google BigTable – General Characteristica

Examples ‘Massive data stored’ for Web indexing,

Google Earth, and Google Finance

Characteristica Scalability: Bigtable is designed to ‘reliably scale to PBs of data’ and

thousands of machines for processing this data ‘String-based approach’ to support large quantities of texts Shares many ‘implementation strategies’ with traditional databases Deployed on top of distributed file system

e.g. Google file system ~HDFS (cf. Lecture 5)


BigTable is a distributed storage system for managing structured data Designed to scale to a very large size (e.g. PBs) of data across thousands of commodity servers BigTable is based on a flexible data model that enables dynamic control over data layout/format

[22] Google BigTable

18 / 55

Google BigTable – Data Model


Sparse, distributed, persistent ‘multidimensional sorted map’ Map is indexed by: ‘row key, column key, and a timestamp key’

(row:string, column:string, time:int64) string Map values are ‘uninterpreted’ array of bytes

Example: ‘Storing Web pages’ E.g. ‘PageRank needs links to this page’, cf. Lecture 3

Row name: Reversed URL Contents column: page contents (here 3 timestamps/row)

Anchor column(s) content:text of any anchorslinking to thepage (here the CNN page referenced from cnnsi.com and my.look.ca)

Anchor column(s) name: URIs of references to this page

[22] Google BigTable

[23] Parallel Programmingwith Spark

19 / 55

GAE Services – Understanding SQL vs. NoSQL

Simple NoSQL example (e.g. Google Datastore & Google Bigtable) E.g. store flexible key - value pairs vs. Structured Query Language (SQL) use


SQL and NoSQL databases are two complementary approaches and applications use both of them

SQL Databases Good for ‘structured data’ Relational Database

Management Systems SQL DB schema with well-

defined columns/rows

NoSQL Databases Good for ‘unstructured data’ Easy to deploy & implement Often geographically distributed Designed with ‘no schemas’ Low data consistency requirements

20 / 55

NoSQL Concepts for Databases

Wide variety of emerging NoSQL databases for ‘specific data’ Most commonly agreed classification based on data models (below) More and more NoSQL data-base implementation become available Several closed source, many open source databases available

Column-based E.g. Apache Hbase

Document-based E.g. MongoDB, CouchDB

Key-Value-based E.g. Apache Cassandra

Graph-based E.g. Neo4J


Tools: NoSQL Databases

Many ‘NoSQL databases’ are emerging in context of map-reduce Based on ‘key-value pair’ designs instead of ‘traditional well-defined tables’

E.g. Apache HBase ‘Hadoop database’ Distributed, scalable, ‘big data store’:

‘billions of rows X, millions of columns’ Provides ‘BigTable-like’ capabilities on top of

Hadoop and HDFS (like Google BigTable on top of Google FS approach) Offers ‘column-oriented store’, but different to normal database (i.e. NoSQL) Supports better ‘write-intensive’ applications (i.e. ‘read-oriented’) Enables ‘transactions (update,insert, delete)‘

on top of Apache Hadoop (cf. Lecture 5)


[24] HBASE Web Page

Big Data Analytics Tools go beyond MPI or map-reduce/HDFS and include NoSQL databases, relational databases, datastream tools, metadata tools, visualization tools, and many others

The usage of which tools mostly depends on the application(s) that should be supported

22 / 55

Apache Hbase Details

Data model design (cf. BigTable design earlier) Columns can be added anytime ‘w/o announcement’ to ‘Columnfamilies’ Allows seamless adding of more ‘links to this page’

Conceptual View

Physical View (no empty fields are stored, difference to RDMS)


[24] HBASE Web Page

[24] HBASE Web Page

23 / 55

MongoDB NoSQL Database

Document-based Example: MongoDB Selected features: index support, querying, open source, C++, … Data Model: Simple & dynamic schemas with ‘JSON-Style documents’

(JSON = JavaScript Object Notation)


[25] MongoDB Web Page

24 / 55

Apache Cassandra NoSQL Database

Key-Value-based Example: Cassandra (in use at Twitter, eBay, Netflix, etc.) Selected features: decentralized – every node equal, elastic, replication Approach: Using primary keys and ‘hash values’ to determine placement


[26] Cassandra Web Page

25 / 55

(compare to relational database that would have lots of missing entries,essentially being especially for social media applications a sparse table)

GAE Sign-Up Process & Payments – Revisited

Account Identity via a Google account Address, Phone, Email Company or private person Tax information

Payment(!) Credit Card & monthly payments

Every service has its price Example: GAE

Cloud datastore


(informally known as ‘swipe the credit card‘)

26 / 55

PAAS Provider Example – GAE Dashboard & Projects


PAAS Provider Example – GAE Developer Resources


GAE Software Development Kit (SDK)

Cloud Platform SDK Downloadable in four programmig languages (Java, PHP, Python, Go) Use services of an underlying remote infrastructure provided by Google

(without the real need to think and care about the infrastructure itself) Enables to focus on core business, e.g. getting users for apps (not IT!) Java example:


[8] GAE SDK Java Download

The GAE SDK is downloadable in four programming languages (Java, PHP, Python, Go) and provides a local development server as well as tools for deploying and managing GAE applications

29 / 55

GAE SDK – Java Doc Packages Example


(e.g. API to access Google Cloud Datastore)

30 / 55

Payment Models & GAE Pricing


[Video] Google App Engine


[9] YouTube, ‘Getting started with App Engine’

32 / 55

Advanced PAAS Topics


Internet Cloud Systems – Examples from Every Day Life

Selected Cloud Systems (aka ‘Clouds‘) known today Google AppEngine Amazon Web Service Facebook SalesForce.com Rackspace IBM Bluemix Enomaly

Cloud systems play an increasingly important role in upgrading traditional Web services and Internet applications A wide variety of innovative & emerging (start-up) companies use Clouds


Large-scale Internet applications have significantly enhanced the quality of life in society today[1] Distributed & Cloud Computing Book

(massive computing & storage & applications)

(massive computing & storage)

(online social networking & advertisement)

(customer relationship management)

(managed cloud provider & hosting)

(cloud platform)

(elastic computing cloud)

34 / 55

Startup Voodoo Example – Quiz Run Mobile Game App

Business Case Entertaining and addictive game on quizzes Answer quiz questions fast among friends Competition: best list, win medals & coins Revenue: In-app purchases & ads

Challenges Sustains rapidly growing business (scalability for 3 mio players & big data) E.g. app grew from 100 to 60.000 daily active users in less than a month Game performance is criticial (many competitors), good servers required

Approach Use Google Cloud with GAE, Google Datastore, Google Big Query


[10] Google Play Quiz Run

[11] Voodoo Case Study

Startup company Voodoo publishes the Quiz Run game app by using the GAE as main game server GAE connects to mobile game apps via Web services & app uses Google BigQuery & AppStats GAE connects to the backend via Objectify that is a simple interface to the Google Datastore

35 / 55

Google Cloud Example – Data Analytics & Machine Learning


The GAE services re-use other existing Big Data services of the Google Cloud Platform through welldefined interfaces such as the Google BigQuery or Google Dataproc for Apache Spark & Hadoop

Google Dataproc enables scalable & automated cluster management for Apache Spark with quick deployment, logging, and monitoring in order to focus on data analysis and not on infrastructure

(cf. Lecture 3 & 5)

36 / 55

Google Big Query

Voodoo application use case Performs analytics in order to help the company cut down on user churn Avoid another analysis tool in order to avoid data exports from storage Work on algorithms that predict when a user is about to abondon app If this is known in time form usage data, Voodoo gives reasons to stay

Google Big Query Key functionalities are fast data scans (e.g. TB in seconds, PB in minutes) Easily load data from Google infrastructure (e.g. data from IAAS Google

Cloud Storage or PAAS Google Cloud DataStore) Automatically encrypts and replicates data UI offers interactive analysis

of massive datasets


Google Big Query is a fast solution for large-scale big data analytics to find meaningful insights Google Big Query enables users to take advantage of the power of Google search engine for data

[12] Google Big Query

37 / 55

GAE SDK – AppStats Library

Voodoo application use case App should be transparent to costs with high performance & efficiency Need of a mechanism to understand whether games are stores efficiently Understanding whether the app is making unnecessary network traffic Benefits: Boosting performance: revamp game storage to improve refresh

rates and avoid slow retrieval times for high number of users Revenue: Cut costs while boosting performance

AppStat Library Profiling of remote procedure calls (RPC, Web service message exchange) GAE RPC is a roundtrip network call between application and GAE APIs Enables to obtain an overview of costs due to RPC profiling


GAE SDK library AppStats is an API call profiling tool that measures peformance of Apps GAE SDK library AppStats enables to observe the exact cost of any request issued to a server

[13] GAE AppStats

38 / 55

Rovio Games Example – Angry Birds Game App

Business Case Angry Birds destroy different pigs Each bird has unique talents Revenue: In-app purchases & ads

Challenges More than 140 million downloads & users Robust capabilities and scalability to deliver

a superior user experience for various apps

Approach Rovio Web games tend to be popular immediately (no scaling over time) Use Google Cloud Platform: GAE, Memcache API, Google Cloud Datastore


[14] Google PlayAngry Birds

[15] Rovio Case Study

The company Rovio publishes the Angry Birds game app by using the GAE as main game server GAE is able to instantly launch as many servers as required for highly popular Angry Birds variants GAE manages all the servers w/o much maintenance enabling 1-2 developers per Angry Bird app

39 / 55

GAE MemCache

Rovio application use case Apps often face ‘similiar situations‘ that needs the same data Boost performance by providing tempory high-speed

data access to same query result data in the Angry Birds app High performance scalable Web applications such as Angry Birds

could use a distributed in-memory data cache in front persistent storage

GAE Memcache Idea: check the memcache first for results of older queries Only perform the datastore query if the results are absent or expired Session data, user preferences, and other data returned by queries for

Apps are good candidates for caching and can benefit from Memcache Other example are temporary values (e.g. rain yes/no, might disappear)


[16] GAE Memcache

GAE MemCache enables speed up of common datastore queries by using a global cache approach GAE MemCache can cache the results if many requests make same query with same parameters

40 / 55

Google Cloud Example – Data Analytics & Machine Learning


The Google Cloud Artificial Intelligence (AI) & Machine Learning (ML) PAAS services offer a broadrange of standard machine learning tasks & functionalities that can be integrated into applications

The Google AI & ML PAAS services are called Cloud AutoML, Cloud TPU, Cloud Machine Learning Engine, and Cloud Talent Solution – however many of the services are still in Beta development

(cf. Lecture 3 & 5)

Google Cloud Platform machine learning services enable PAAS developers to easily use algorithms41 / 55

[20] Lego Bricks [34] Google Cloud Analytics & Machine Learning

Google Cloud Example – Cloud AutoML Products

Vision Service Offers pre-trained machine learning models via an API Provides ability to build custom machine learning models

Natural Language Service Machine learning API to derive insights from unstructured texts Extract information mentioned in text documents, news articles, blogs

Translation Service API that enables dynamic translations between languages Offers also language detection functionality


The PAAS concept of Google Cloud AutoML is totrain machine learning models with minimum effort& machine learning expertise

Google Cloud AutoML offers three concrete servicesfor Vision, Natural Language analysis and translation

[27] Google Cloud AutoML Products

42 / 55

Google Cloud Example – Cloud Tensor Processing Unit

Hardware-driven PAAS Service Offers accelerator technology to speed up

machine learning workloads on Google Clouds Enables TensorFlow (cf. Lecture 6)

deep learning models to run fast on a specialized chip

Provides ‘Cloud Pods’ with high-speed network interconnects


The concept of the Google Cloud Tensor Processing Unit (TPU) isto train & run machine learning models faster using specializedhardware chip as Application-specific Integrated Circuit (ASIC)

Google Cloud TPU is very useful to train & run computationallydemanding deep learning models with TensorFlow in clouds

[28] Google Cloud TPU

(land coverdetection,cf. PracticalLecture 6.1)

(convolutional neural network deeplearning model using tensors, cf. Lecture 6)

[29] Big Data Tips, What is a Tensor?

[30] Tensorflow

43 / 55

Google Cloud Example – Cloud Machine Learning Engine

Flexible Service for Machine Learning Training and prediction services that can be

integrated into a data-intensive application

Supports several machine & deep learning frameworks TensorFlow, Keras, scikit-learn, etc. XGBoost provides a parallel tree

boosting machine learningmodel framework that is oftenused & supports parallel execution(e.g. Hadoop, cf. Lecture 5)


The concept of the Google Cloud Machine Learning Engine is toenable developers to build large-scale machine learning models(e.g. regression, image classification, etc.) using cloud resources

The Cloud Machine Learning Engine offers managed PAAS servicesfor training machine learning models and using them for prediction

[31] Google Cloud Machine Learning Engine

[32] XGBoost Web page

44 / 55

Microsoft Azure Example – Artificial Intelligence Overview


The concept of the Microsoft Azure Cloud services for machine & deep learning include severalPAAS-driven tools, frameworks, & infrastructure for Artificial Intelligence (AI) coding & modeling

[32] Microsoft Azure Artificial Intelligence

45 / 55

AWS Cloud Example – Machine Learning Overview


The concept of AWS Cloud PAAS services include APIs for several Vision services, Conversationalchatbots, and various Language Services that include translations & speach recognition

[33] AWS Cloud Machine Learning

46 / 55

Other PAAS Providers & Disadvantages


[1] Distributed & Cloud Computing Book

PAAS disadvantages are that application developers are being locked in to a certain platform (aka ‘vendor lock‘) and that they may not be able to use known tools (e.g. ORACLE DB) with services

47 / 55

PAAS Provider: Manjrasoft Aneka – Standards Example


[17] ManjrasoftWeb page

(neutral key playerfrom Australia)

(Standard ECMA-334:is C# languagespecification)

(ECMA International:European Association

for standardizinginformation andcommunication

systems)

Use of standard programming languages and interface standards avoids or reduces vendor-locks

48 / 55

[Video] From Zero to ML on Google Cloud


[18] YouTube, ‘From Zero to ML on Google Cloud Platform (Cloud Next '18)’

49 / 55

Lecture Bibliography


Lecture Bibliography (1)

[1] K. Hwang, G. C. Fox, J. J. Dongarra, ‘Distributed and Cloud Computing’, Book, Online: http://store.elsevier.com/product.jsp?locale=en_EU&isbn=9780128002049

[2] Key-based Authentication in SSH, Online: http://www.oracle.com/webfolder/technetwork/tutorials/obe/cloud/dbaas/obe_dbaas_connecting_to_an_instance/images/key%20based%20authentication.jpg

[3] Accenture, Hadoop Deployment Comparison Study, Online: http://www.accenture.com/SiteCollectionDocuments/PDF/Accenture-Hadoop-Deployment-Comparison-Study.pdf

[4] Wikipedia, ‚Cloud Computing‘, Online: http://de.wikipedia.org/wiki/Cloud-Computing

[5] Google App Engine Web page, Online: https://cloud.google.com/appengine/

[6] Google Cloud Platform Case Study, ‘Philipps‘, Online: https://cloud.google.com/customers/philips/

[7] Google Cloud Datastore, Online: https://cloud.google.com/datastore

[8] Google App Engine Java SDK Download, Online: https://cloud.google.com/appengine/docs/java/download

[9] YouTube video, ‘Getting Started with App Engine‘, Online: https://www.youtube.com/watch?v=-7ml51INtEM

[10] Google Play Quiz Run App, Online: https://play.google.com/store/apps/details?id=air.com.voodoo.quizrun&hl=en



[11] Google Cloud Platform Case Study‚ ‘Voodoo‘, Online: https://cloud.google.com/customers/voodoo/

[12] Google Big Query, Online: https://cloud.google.com/bigquery/

[13] Google App Engine, ‘GAE AppStats‘, Online: https://cloud.google.com/appengine/docs/java/tools/appstats

[14] Google Play Angry Birds, Online: https://play.google.com/store/apps/details?id=com.rovio.angrybirds&hl=en

[15] Google Cloud Platform Case Study‚ ‘Rovio‘, Online: https://cloud.google.com/customers/rovio

[16] Google App Engine, ‘GAE Memcache‘, Online: https://cloud.google.com/appengine/docs/python/memcache/

[17] Manjrasoft Web page, Online: http://www.manjrasoft.com/aneka_architecture.html

[18] YouTube video, ‘From Zero to ML on Google Cloud Platform (Cloud Next '18)‘, Online: https://www.youtube.com/watch?v=QU7_eU8HzAQ

[19] Google Cloud – Compute Products,Online: https://cloud.google.com/products/compute/

[20] Lego Bricks, Online: https://commons.wikimedia.org/wiki/File:Lego_dimensions.svg



[21] Wikipedia, ’NoSQL’, Online: http://en.wikipedia.org/wiki/NoSQL

[22] Fay Chang, Jeffrey Dean, Sanjay Ghemawat, Wilson C. Hsieh, Deborah A. Wallach, Mike Burrows, TusharChandra, Andrew Fikes, and Robert E. Gruber, ‘Bigtable: A Distributed Storage System for Structured Data’,Online: http://static.usenix.org/events/osdi06/tech/chang/chang_html/?em_x=22

[23] M. Zaharia, ‘Parallel Programming with Spark‘, 2013 [24] Apache HBase,

Online: http://hbase.apache.org/ [25] MongoDB,

Online: http://www.mongodb.org/ [26] Apache Cassandra,

Online: http://cassandra.apache.org/ [27] Google Cloud AutoML Products,

Online: https://cloud.google.com/automl/ [28] Google Cloud TPU Products,

Online: https://cloud.google.com/tpu/ [29] Big Data Tips, ‘What is a Tensor?‘,

Online: http://www.big-data.tips/what-is-a-tensor [30] Tensorflow Deep Learning Framework,

Online: https://www.tensorflow.org/



[31] XGBoost Web page,Online: https://xgboost.readthedocs.io/en/latest/

[32] Microsoft Azure Artificial Intelligence,Online: https://azure.microsoft.com/en-us/overview/ai-platform/

[33] AWS Cloud Machine Learning,Online: https://aws.amazon.com/machine-learning/?nc1=h_ls

[34] Google Cloud Analytics & Machine Learning,Online: https://cloud.google.com/products/#analytics-and-machine-learning
