1©Cloudera,Inc.Allrightsreserved.
AshishSingh|So:wareEngineer,Cloudera
Authoriza>oninApacheKaAa
2©Cloudera,Inc.Allrightsreserved.
• So:wareEngineer@Cloudera• ContributedtoKaAa,Sentry,HiveandParquet• UsedtoworkinHPC• @singhasdev
AboutMe
3©Cloudera,Inc.Allrightsreserved.
4©Cloudera,Inc.Allrightsreserved.
AboutKaAa
• Publish/SubscribeMessagingSystem• Highthroughput(100’sofkmessages/sec)• Lowlatency(sub-secondtolowseconds)• Fault-tolerant(ReplicatedandDistributed)• Supportsagnos>cmessaging• Standardizesformatanddelivery• Hugecommunity
5©Cloudera,Inc.Allrightsreserved.
ArchitectureProducer
Consumer Consumer
Producers
KaAaCluster
Consumers
Broker Broker Broker Broker
Producer
Zookeeper
6©Cloudera,Inc.Allrightsreserved.
Authoriza>on
7©Cloudera,Inc.Allrightsreserved.
Authoriza>on
Authoriza>onisthefunc>onofspecifyingaccessrightstoresourcesrelatedtoinforma>onsecurityandcomputersecurityingeneralandtoaccesscontrolinpar>cular.Moreformally,"toauthorize"istodefineanaccesspolicy.–Wikipedia
8©Cloudera,Inc.Allrightsreserved.
Authoriza>on
Authoriza>onisthefunc>onofspecifyingaccessrightstoresourcesrelatedtoinforma>onsecurityandcomputersecurityingeneralandtoaccesscontrolinpar>cular.Moreformally,"toauthorize"istodefineanaccesspolicy.–Wikipedia
9©Cloudera,Inc.Allrightsreserved.
AccessPolicy
WHOcanperformWHATac>ononaRESOURCE?
10©Cloudera,Inc.Allrightsreserved.
AccessPolicy
WHOcanperformWHATac>ononaRESOURCE?
11©Cloudera,Inc.Allrightsreserved.
Authoriza>oninApacheKaAa
12©Cloudera,Inc.Allrightsreserved.
AccessPolicy
WHOcanperformWHATac>ononaRESOURCE?
13©Cloudera,Inc.Allrightsreserved.
AccessPolicy
WHOcanorcannotperformWHATac>ononaRESOURCEfromWHERE?
14©Cloudera,Inc.Allrightsreserved.
AccessPolicy
WHOcanorcannotperformWHATac>ononaRESOURCEfromWHERE?• KaAausesabinaryprotocoloverTCP.• TheprotocoldefinesallAPIsasrequestresponsemessagepairs.• RequestsaresentthroughaRequestchannela:erasessionisestablished.• SessioncontainsPrincipalandHost.• Principalisoftheform<PrincipalType>:<PrincipalName>,.e.g.,User:foo,Group:analyst,etc.• Hosthasinfoonwheretherequesthasoriginatedfrom.• ProvidesIP-Filtering.
15©Cloudera,Inc.Allrightsreserved.
AccessPolicy
WHOcanorcannotperformWHATac>ononaRESOURCEfromWHERE?• PermissionTypes.• Allow• Deny
16©Cloudera,Inc.Allrightsreserved.
AccessPolicy
WHOcanorcannotperformWHATac>ononaRESOURCEfromWHERE?• Opera>onssupportedinKaAa.• Read• Write• Create• Delete• Alter• Describe• ClusterAc>on
17©Cloudera,Inc.Allrightsreserved.
AccessPolicy
WHOcanorcannotperformWHATac>ononaRESOURCEfromWHERE?• ResourceTypesinKaAa.• Cluster• Topic• ConsumerGroup
18©Cloudera,Inc.Allrightsreserved.
AccessPolicy
• AccessPolicyinKaAaisrepresentedasACL,AccessControlList.• ACLinKaAaiscomposedofthefollowing.• SetofPrincipals.• Permissiontype,i.e.,AlloworDeny.• SetofHosts.• SetofOpera>ons.
19©Cloudera,Inc.Allrightsreserved.
KaAa’sDefaultAuthorizerSimpleAclAuthorizer
20©Cloudera,Inc.Allrightsreserved.
SimpleAclAuthorizer
• Outoftheboximplementa>onoftheKaAaAuthorizer.• Selfcontainedandnodependencieswithanyothervendororproviders.• Ituseszookeeperasthestoragelayerforacls.ACLsarestoredinJSONformatdescribedunder/kaAa-acls/resource-type/<resource-name>.• UsesCachingtoavoidgoingtoZKforeachrequest.• DenytakesprecedenceoverAllowincompe>ngACLs.• WhennoACLisaoachedtoaresource,useconfigallow.everyone.if.no.acl.found.• WhenanyACLisaoachedtoaresourceonlyusersthatareintheallowedlisthaveaccess.AlluserswithnoexplicitallowACLsaredeniedaccessbydefault.• READorWRITEpermission=>DESCRIBEOpera>on.
21©Cloudera,Inc.Allrightsreserved.
SimpleAclAuthorizer–NotsoSimple
• OnlysupportsUserprincipal• PRoutforGroupPrincipalforsome>me.
• Nowaytouseusergroupmappingfromexternalservices,like,LDAP,AD,etc.• VeryKaAaspecificimplementa>on.• Notscalable.• HaszNodesizelimita>ons,defaultandrecommendedinonly1MB.• Concurrencyissues.
• Notproduc>onready.
22©Cloudera,Inc.Allrightsreserved.
23©Cloudera,Inc.Allrightsreserved.
Sentry
• Providesunifiedrolebasedauthoriza>onforvariouscomponents.• Hive• Impala• HDFS• Sqoop• KaAa(ohyea!)
24©Cloudera,Inc.Allrightsreserved.
RBAC
• RoleBasedAuthoriza>onControl,RBAC,isapowerfulmechanismtomanageauthoriza>onforalargesetofusersanddataobjectsinatypicalenterprise.• Assigningprivilegestoauser.• Privilege->User• Privilege->Group->User• Privilege->Role->Group->User
• UsergroupmappingisconfigurableandcancomefromShellorexternalsystems,like,LDAP,AD,etc.
25©Cloudera,Inc.Allrightsreserved.
RBAC
CFO
CustomerDataDataEngineers
Analysts EnrichedData
SalesPredic>ons
26©Cloudera,Inc.Allrightsreserved.
RBAC
CFO
CustomerDataDataEngineers
Analysts EnrichedData
SalesPredic>ons
• DataEngineerscanREADfromCustomerData.• DataEngineerscanWRITEtoEnrichedData.• AnalystscanREADfromEnrichedData.• AnalystscanWritetoSalesPredic>on.• CFOcanREADfromSalesPredic>on.
27©Cloudera,Inc.Allrightsreserved.
RBAC
CFO
CustomerDataDataEngineers
Analysts EnrichedData
SalesPredic>ons
• Roles:• ReadCustomerData=>READfromCustomerData• WriteEnrichedData=>WRITEtoEnrichedData• ReadEnrichedData=>READfromEnrichedData• WriteSalesPredic>ons=>WRITEtoSalesPredic>ons• ReadSalesPredic>ons=>READfromSalesPredic>ons• AllRead=>READfromCustomer,EnrichedandSalesPredic>ons
• RolestoGroups:• DataEngineers=>ReadCustomerData,ReadEnrichedData,
WriteEnrichedData• Analysts=>ReadEnrichedData,ReadSalesPredic>ons,
WriteSalesPredic>ons• CFO=>ReadSalesPredic>ons
28©Cloudera,Inc.Allrightsreserved.
RBAC
CFO
CustomerDataDataEngineers
Analysts EnrichedData
SalesPredic>ons
• JoejustjoinedasDataEngineers• JustaddJoetoDataEngineersgroup• JoegetsallDataEngineersprivileges
29©Cloudera,Inc.Allrightsreserved.
RBAC
CFO
CustomerDataDataEngineers
Analysts EnrichedData
SalesPredic>ons
• Newteamaddedtomaintaindatalineage.• Justaddrolestoit,noneedredefine
privileges.• Auditors=>ReadAll
Auditors
30©Cloudera,Inc.Allrightsreserved.
31©Cloudera,Inc.Allrightsreserved.
SentryKaAaAuthorizer
• Bringsrolebasedauthoriza>oncontroltoKaAa.• Useusergroupmappingsfromexternalsystems,like,LDAP,AD,etc.• Scalablearchitecture.• Unifiedauthoriza>oncontrolacrossvariousdatainfrastructurecomponents.
32©Cloudera,Inc.Allrightsreserved.
SentryKaAaAuthorizer
Broker
Broker
Broker
Producer
Consumer
Sentry
1. Clientauthen>c
ateswithBroker.
33©Cloudera,Inc.Allrightsreserved.
SentryKaAaAuthorizer
Broker
Broker
Broker
Producer
Consumer
Sentry
1. Clientauthen>c
ateswithBroker.
2. Clientsendsreq
uesttoaBroker.
34©Cloudera,Inc.Allrightsreserved.
SentryKaAaAuthorizer
Broker
Broker
Broker
Producer
Consumer
Sentry
1. Clientauthen>c
ateswithBroker.
2. Clientsendsreq
uesttoaBroker.
35©Cloudera,Inc.Allrightsreserved.
SentryKaAaAuthorizer
Broker
Broker
Broker
Producer
Consumer
Sentry
1. Clientauthen>c
ateswithBroker.
2. Clientsendsreq
uesttoaBroker.
36©Cloudera,Inc.Allrightsreserved.
SentryKaAaAuthorizer
Broker
Broker
Broker
Producer
Consumer
Sentry
1. Clientauthen>c
ateswithBroker.
2. Clientsendsreq
uesttoaBroker.
5.BrokersendsNo
tauthorizederror
codeiftherequest
isnotauthorized.
Otherwisesendsa
ppropriaterespons
e
fortherequest.
37©Cloudera,Inc.Allrightsreserved.
Demo
38©Cloudera,Inc.Allrightsreserved.
[email protected]@singhasdev