spark will replace hadoop ! know why

http://www.edureka.co/apache-spark-scala-training

Spark will replace Hadoop ! Know Why ?

Slide 2Slide 2 http://www.edureka.co/apache-spark-scala-training

At the end of the session, you will be able to:

Understand Why Learn Spark?

Know Advantages of Spark & its Survey for 2015

Discover Spark Career Path

Understand how Companies are using Spark?

Agenda


Why Spark?


Rise of Big Data

By 2020, IDC (International Data Corporation) predicts the number will have reached 40,000 EB, or 40 Zettabytes (ZB)

The world’s information is doubling every two years. By 2020, there will be 5,200 GB of data for every person on

Earth.

0

1000

2000

3000

4000

5000

6000

7000

2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

Unstructured Data

Structured Data Un-structured Data


Application of Big Data

Source: Twitter


Application of Big Data


Hadoop is not Enough!

Limitations:

Conclusion:

Real-time Processing

Not Fast Enough

Hadoop MapReduce is Limited to Batch Processing. Real-time processing was a big “No” in Hadoop

Hadoop MapReduce is fast but not fast enough

It is essential and can be achieved using Spark!


Spark Survey and its Advantages


Spark Survey 2015!

Source: Typesafe


Advantages of Spark

Ease of Use

Generality

Runs Everywhere

100x faster than MR


Feature Comparision

Fast 100x faster than MapReduce

Batch Processing Batch and Real-time Processing

Stores Data on Disk Stores Data in Memory

OpenSource OpenSource

Written in Java Written in Scala

Hadoop MapReduce HADOOP Spark

Source: Databrix


Spark Features/Modules in Demand

Source: Typesafe


New Features in 2015

Data Frames

• Similar API to data frames in R and Pandas• Automatically optimised via Spark SQL• Released in Spark 1.3

SparkR

• Released in Spark 1.4• Exposes DataFrames, RDD’s & ML library in R

Machine Learning Pipelines

• High Level API• Featurization• Evaluation • Model Tuning

External Data Sources

• Platform API to plug Data-Sources into Spark• Pushes logic into sources

Source: Databrix


Spark Career Path


Job Roles & Industry Focus

Source: Typesafe


Salary Trends


Major Companies Using Hadoop


Industry Adoption

Source: Typesafe


How Companies are using Spark?


General Business Goals

Source: Typesafe


Demo


The Big Question!

Is Spark going to replace Hadoop?


The Big Question!

Is Spark going to replace Hadoop?

Answer – Yes, Spark will be used on top of Hadoop and replace MapReduce

Reasons:

1. Hadoop MapReduce cannot handle real-time processing 2. Hadoop MapReduce is slower than Hadoop Spark3. With rise of IOT, Spark is a must

Questions

Slide 24 http://www.edureka.co/apache-spark-scala-training

Your feedback is important to us, be it a compliment, a suggestion or a complaint. It helps us to make the course better!

Please spare few minutes to take the survey after the webinar.


Survey

spark will replace hadoop ! know why

Technology