Authorization in Apache Kafka - Seattle Kafka Meetup - Ashish Singh

Download Authorization in Apache Kafka - Seattle Kafka Meetup - Ashish Singh

Post on 14-Apr-2017

157 views

Category:

Documents

4 download

TRANSCRIPT

  • 1Cloudera,Inc.Allrightsreserved.

    AshishSingh|So:wareEngineer,Cloudera

    Authoriza>oninApacheKaAa

  • 2Cloudera,Inc.Allrightsreserved.

    So:wareEngineer@ClouderaContributedtoKaAa,Sentry,HiveandParquetUsedtoworkinHPC@singhasdev

    AboutMe

  • 3Cloudera,Inc.Allrightsreserved.

  • 4Cloudera,Inc.Allrightsreserved.

    AboutKaAa

    Publish/SubscribeMessagingSystemHighthroughput(100sofkmessages/sec) Lowlatency(sub-secondtolowseconds) Fault-tolerant(ReplicatedandDistributed) Supportsagnos>cmessaging StandardizesformatanddeliveryHugecommunity

  • 5Cloudera,Inc.Allrightsreserved.

    ArchitectureProducer

    Consumer Consumer

    Producers

    KaAaCluster

    Consumers

    Broker Broker Broker Broker

    Producer

    Zookeeper

  • 6Cloudera,Inc.Allrightsreserved.

    Authoriza>on

  • 7Cloudera,Inc.Allrightsreserved.

    Authoriza>on

    Authoriza>onisthefunc>onofspecifyingaccessrightstoresourcesrelatedtoinforma>onsecurityandcomputersecurityingeneralandtoaccesscontrolinpar>cular.Moreformally,"toauthorize"istodefineanaccesspolicy.Wikipedia

  • 8Cloudera,Inc.Allrightsreserved.

    Authoriza>on

    Authoriza>onisthefunc>onofspecifyingaccessrightstoresourcesrelatedtoinforma>onsecurityandcomputersecurityingeneralandtoaccesscontrolinpar>cular.Moreformally,"toauthorize"istodefineanaccesspolicy.Wikipedia

  • 9Cloudera,Inc.Allrightsreserved.

    AccessPolicy

    WHOcanperformWHATac>ononaRESOURCE?

  • 10Cloudera,Inc.Allrightsreserved.

    AccessPolicy

    WHOcanperformWHATac>ononaRESOURCE?

  • 11Cloudera,Inc.Allrightsreserved.

    Authoriza>oninApacheKaAa

  • 12Cloudera,Inc.Allrightsreserved.

    AccessPolicy

    WHOcanperformWHATac>ononaRESOURCE?

  • 13Cloudera,Inc.Allrightsreserved.

    AccessPolicy

    WHOcanorcannotperformWHATac>ononaRESOURCEfromWHERE?

  • 14Cloudera,Inc.Allrightsreserved.

    AccessPolicy

    WHOcanorcannotperformWHATac>ononaRESOURCEfromWHERE?KaAausesabinaryprotocoloverTCP.TheprotocoldefinesallAPIsasrequestresponsemessagepairs.RequestsaresentthroughaRequestchannela:erasessionisestablished. SessioncontainsPrincipalandHost.Principalisoftheform:,.e.g.,User:foo,Group:analyst,etc.Hosthasinfoonwheretherequesthasoriginatedfrom. ProvidesIP-Filtering.

  • 15Cloudera,Inc.Allrightsreserved.

    AccessPolicy

    WHOcanorcannotperformWHATac>ononaRESOURCEfromWHERE?PermissionTypes.AllowDeny

  • 16Cloudera,Inc.Allrightsreserved.

    AccessPolicy

    WHOcanorcannotperformWHATac>ononaRESOURCEfromWHERE?Opera>onssupportedinKaAa.ReadWriteCreateDeleteAlterDescribeClusterAc>on

  • 17Cloudera,Inc.Allrightsreserved.

    AccessPolicy

    WHOcanorcannotperformWHATac>ononaRESOURCEfromWHERE?ResourceTypesinKaAa.ClusterTopicConsumerGroup

  • 18Cloudera,Inc.Allrightsreserved.

    AccessPolicy

    AccessPolicyinKaAaisrepresentedasACL,AccessControlList.ACLinKaAaiscomposedofthefollowing.SetofPrincipals.Permissiontype,i.e.,AlloworDeny.SetofHosts.SetofOpera>ons.

  • 19Cloudera,Inc.Allrightsreserved.

    KaAasDefaultAuthorizerSimpleAclAuthorizer

  • 20Cloudera,Inc.Allrightsreserved.

    SimpleAclAuthorizer

    Outoftheboximplementa>onoftheKaAaAuthorizer. Selfcontainedandnodependencieswithanyothervendororproviders. Ituseszookeeperasthestoragelayerforacls.ACLsarestoredinJSONformatdescribedunder/kaAa-acls/resource-type/.UsesCachingtoavoidgoingtoZKforeachrequest.DenytakesprecedenceoverAllowincompe>ngACLs.WhennoACLisaoachedtoaresource,useconfigallow.everyone.if.no.acl.found.WhenanyACLisaoachedtoaresourceonlyusersthatareintheallowedlisthaveaccess.AlluserswithnoexplicitallowACLsaredeniedaccessbydefault.READorWRITEpermission=>DESCRIBEOpera>on.

  • 21Cloudera,Inc.Allrightsreserved.

    SimpleAclAuthorizerNotsoSimple

    OnlysupportsUserprincipalPRoutforGroupPrincipalforsome>me.

    Nowaytouseusergroupmappingfromexternalservices,like,LDAP,AD,etc.VeryKaAaspecificimplementa>on.Notscalable.HaszNodesizelimita>ons,defaultandrecommendedinonly1MB.Concurrencyissues.

    Notproduc>onready.

  • 22Cloudera,Inc.Allrightsreserved.

  • 23Cloudera,Inc.Allrightsreserved.

    Sentry

    Providesunifiedrolebasedauthoriza>onforvariouscomponents.Hive ImpalaHDFSSqoopKaAa(ohyea!)

  • 24Cloudera,Inc.Allrightsreserved.

    RBAC

    RoleBasedAuthoriza>onControl,RBAC,isapowerfulmechanismtomanageauthoriza>onforalargesetofusersanddataobjectsinatypicalenterprise.Assigningprivilegestoauser.Privilege->UserPrivilege->Group->UserPrivilege->Role->Group->User

    UsergroupmappingisconfigurableandcancomefromShellorexternalsystems,like,LDAP,AD,etc.

  • 25Cloudera,Inc.Allrightsreserved.

    RBAC

    CFO

    CustomerDataDataEngineers

    Analysts EnrichedData

    SalesPredic>ons

  • 26Cloudera,Inc.Allrightsreserved.

    RBAC

    CFO

    CustomerDataDataEngineers

    Analysts EnrichedData

    SalesPredic>ons

    DataEngineerscanREADfromCustomerData. DataEngineerscanWRITEtoEnrichedData. AnalystscanREADfromEnrichedData. AnalystscanWritetoSalesPredic>on. CFOcanREADfromSalesPredic>on.

  • 27Cloudera,Inc.Allrightsreserved.

    RBAC

    CFO

    CustomerDataDataEngineers

    Analysts EnrichedData

    SalesPredic>ons

    Roles: ReadCustomerData=>READfromCustomerData WriteEnrichedData=>WRITEtoEnrichedData ReadEnrichedData=>READfromEnrichedData WriteSalesPredic>ons=>WRITEtoSalesPredic>ons ReadSalesPredic>ons=>READfromSalesPredic>ons AllRead=>READfromCustomer,EnrichedandSalesPredic>ons

    RolestoGroups: DataEngineers=>ReadCustomerData,ReadEnrichedData,

    WriteEnrichedData Analysts=>ReadEnrichedData,ReadSalesPredic>ons,

    WriteSalesPredic>ons CFO=>ReadSalesPredic>ons

  • 28Cloudera,Inc.Allrightsreserved.

    RBAC

    CFO

    CustomerDataDataEngineers

    Analysts EnrichedData

    SalesPredic>ons

    JoejustjoinedasDataEngineers JustaddJoetoDataEngineersgroup JoegetsallDataEngineersprivileges

  • 29Cloudera,Inc.Allrightsreserved.

    RBAC

    CFO

    CustomerDataDataEngineers

    Analysts EnrichedData

    SalesPredic>ons

    Newteamaddedtomaintaindatalineage. Justaddrolestoit,noneedredefine

    privileges. Auditors=>ReadAll

    Auditors

  • 30Cloudera,Inc.Allrightsreserved.

  • 31Cloudera,Inc.Allrightsreserved.

    SentryKaAaAuthorizer

    Bringsrolebasedauthoriza>oncontroltoKaAa.Useusergroupmappingsfromexternalsystems,like,LDAP,AD,etc. Scalablearchitecture.Unifiedauthoriza>oncontrolacrossvariousdatainfrastructurecomponents.

  • 32Cloudera,Inc.Allrightsreserved.

    SentryKaAaAuthorizer

    Broker

    Broker

    Broker

    Producer

    Consumer

    Sentry

    1. Clientauthen>c

    ateswithBroker.

  • 33Cloudera,Inc.Allrightsreserved.

    SentryKaAaAuthorizer

    Broker

    Broker

    Broker

    Producer

    Consumer

    Sentry

    1. Clientauthen>c

    ateswithBroker.

    2. Clientsendsreq

    uesttoaBroker.

  • 34Cloudera,Inc.Allrightsreserved.

    SentryKaAaAuthorizer

    Broker

    Broker

    Broker

    Producer

    Consumer

    Sentry

    1. Clientauthen>c

    ateswithBroker.

    2. Clientsendsreq

    uesttoaBroker.

  • 35Cloudera,Inc.Allrightsreserved.

    SentryKaAaAuthorizer

    Broker

    Broker

    Broker

    Producer

    Consumer

    Sentry

    1. Clientauthen>c

    ateswithBroker.

    2. Clientsendsreq

    uesttoaBroker.

  • 36Cloudera,Inc.Allrightsreserved.

    SentryKaAaAuthorizer

    Broker

    Broker

    Broker

    Producer

    Consumer

    Sentry

    1. Clientauthen>c

    ateswithBroker.

    2. Clientsendsreq

    uesttoaBroker.

    5.BrokersendsNo

    tauthorizederror

    codeiftherequest

    isnotauthorized.

    Otherwisesendsa

    ppropriaterespons

    e

    fortherequest.

  • 37Cloudera,Inc.Allrightsreserved.

    Demo

  • 38Cloudera,Inc.Allrightsreserved.

    ThankyouAshishSinghasingh@cloudera.com@singhasdev