protecting enterprise data in apache hadoop · pdf filethreat: hadoop admin in cluster page 11...

© Hortonworks Inc. 2015

Protecting Enterprise Data in Apache Hadoop

September2015

Page 1

Owen O’Malley owen@hortonworks.com @owen_omalley

© Hortonworks Inc. 2015

Security

Page 2

© Hortonworks Inc. 2015

Security Architecture

Page 3

© Hortonworks Inc. 2015

Attack Vectors

Page 4

© Hortonworks Inc. 2015

Attack Vectors

Page 5

© Hortonworks Inc. 2015

Threat: Accidental Damage

Page 6

© Hortonworks Inc. 2015

Threat: Remote Access

Page 7

© Hortonworks Inc. 2015

Threat: Eavesdropping

Page 8

© Hortonworks Inc. 2015

Threat: User accesses private data

Page 9

© Hortonworks Inc. 2015

Threat: Physical access

Page 10

© Hortonworks Inc. 2015

Threat: Hadoop Admin in Cluster

Page 11

© Hortonworks Inc. 2015

HDFS Encryption

Page 12

© Hortonworks Inc. 2015

KeyProvider API

Page 13

© Hortonworks Inc. 2015

Encryption Scheme

Page 14

© Hortonworks Inc. 2015

Original Hive Architecture

Page 15

© Hortonworks Inc. 2015

Threat: User Accesses DB directly

Page 16

© Hortonworks Inc. 2015

Hive Architecture with Metastore

Page 17

© Hortonworks Inc. 2015

Threat: User Deletes Hive tables

Page 18

© Hortonworks Inc. 2015

Hive Architecture with Storage-Based Auth

Page 19

© Hortonworks Inc. 2015

Threat: User reads private columns

Page 20

© Hortonworks Inc. 2015

Hive Architecture with Hive Server 2

Page 21

© Hortonworks Inc. 2015

Threat: User reads private columns

Page 22

© Hortonworks Inc. 2015

ORC File Layout

Page 23

File Footer

Postscript

Index Data

Row Data

Stripe Footer25

6 M

B St

ripe

Index Data

Row Data

Stripe Footer

256

MB

Strip

e

Index Data

Row Data

Stripe Footer

256

MB

Strip

e

Column 1

Column 2

Column 7

Column 8

Column 3

Column 6

Column 4

Column 5

Column 1

Column 2

Column 7

Column 8

Column 3

Column 6

Column 4

Column 5

Stream 2.1

Stream 2.2

Stream 2.3

Stream 2.4

© Hortonworks Inc. 2015

Threat: User reads hidden values

Page 24

© Hortonworks Inc. 2015

Threat: Shadow Security

Page 25

© Hortonworks Inc. 2015

Resources

Page 26

© Hortonworks Inc. 2015

Other talks

Page 27

© Hortonworks Inc. 2015

Thank You!

Page 28