certification generation - large deployments

11
7/15/2019 Certification Generation - Large Deployments http://slidepdf.com/reader/full/certification-generation-large-deployments 1/11 SailPointBestPractices Certification Generationonlargedeployments NickWellinghoff Purpose Thecertificationgenerationphaseofaprojectisoftenthefirsttimethebusinesswillseetheresultsof monthsofeffort.Itisimportantthisprocessoperatessmoothlyandcanrecoverfromerrors.During smalldeploymentsthisislessofaconcernasthecertificationgenerationcansimplybedeletedand startedagain.However,onlargedeploymentswherejustthegenerationofthecertificationsmaytake multipledaysgivencertaindataconditions,itisimportanttohavemorecontrolovertheprocess.This documentwilloutlineanapproachthatwillgivetheusertheabilitytooptimizegeneration performance,distributesystemloadandrecovergracefullyfromerrors.

Upload: bradders60

Post on 30-Oct-2015

91 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Certification Generation - Large Deployments

7/15/2019 Certification Generation - Large Deployments

http://slidepdf.com/reader/full/certification-generation-large-deployments 1/11

SailPointBestPractices–CertificationGenerationonlargedeployments

NickWellinghoff

PurposeThecertificationgenerationphaseofaprojectisoftenthefirsttimethebusinesswillseetheresultsof

monthsofeffort.Itisimportantthisprocessoperatessmoothlyandcanrecoverfromerrors.During

smalldeploymentsthisislessofaconcernasthecertificationgenerationcansimplybedeletedand

startedagain.However,onlargedeploymentswherejustthegenerationofthecertificationsmaytake

multipledaysgivencertaindataconditions,itisimportanttohavemorecontrolovertheprocess.This

documentwilloutlineanapproachthatwillgivetheusertheabilitytooptimizegeneration

performance,distributesystemloadandrecovergracefullyfromerrors.

Page 2: Certification Generation - Large Deployments

7/15/2019 Certification Generation - Large Deployments

http://slidepdf.com/reader/full/certification-generation-large-deployments 2/11

BestPractices

Page 2 of 11

TableofContentsPurpose........................................................................................................................................................1

Targetaudience...........................................................................................................................................3

Scriptingthecertificationgenerationprocess.............................................................................................3

CreateCertificationDefinitiontemplates.................................................................................................3

Defineacertificationmanagertask.........................................................................................................4

CustomTaskDefinition........................................................................................................................4

DefinetheGenerateCertsTaskExecutor.............................................................................................5

LeveragingtheGenerateCertsTaskExecutor...............................................................................................6

ScenarioOne:Generatemanagercertificationswhilecontrollinguserloadonthewebservercluster.6

ScenarioTwo:Customerdesiresfastercertificationgenerationperformance.......................................6

ScenarioThree:Thecertificationgenerationtaskfailsinproduction.....................................................7

Appendix......................................................................................................................................................9

ExampleimplementationofGenerateCertsTaskExecutor......................................................................9

Page 3: Certification Generation - Large Deployments

7/15/2019 Certification Generation - Large Deployments

http://slidepdf.com/reader/full/certification-generation-large-deployments 3/11

BestPractices

Page 3 of 11

TargetaudienceDeploymentsthathavethefollowingcharacteristicsarethetargetforthisdocument.

•  Clientsthathaveamanagementhierarchythatincludes1,000+members.

•  Clientsgeneratinganycertificationtypewherethenumberofinstancesexceeds1,000.

•  Thehardwarearchitectureisnotcapableofhandlingtheusecaseofalluserswhohaveopen

certificationsloggingonandusingthesystematthesametime.

•  Projectsthathavetighttimelinesandcertificationgenerationneedstobeacceleratedorerrors

needtoberesolvedwithahigherdegreeofgranularitythannormal.

•  Clientsthatdesireafinerlevelofgranularityforerrorreporting.

•  Clientsthathavecomplexcertificationparameterswherethepossibilityofhumanerrorishigh.

ScriptingthecertificationgenerationprocessToachievethestatedgoalsthecertificationconfigurationandexecutionwillbechangedfromaUIand

userdrivenprocesstoaprogrammaticallydrivenprocess.Thissectionwillcoverhowtodothisfora

theoreticalworldwidemanagercertificationrun.Butthesameprinciplesmaybeappliedtoany

certificationtype.

CreateCertificationDefinitionTemplates

NormallywhentheUIisusedtocreateacertificationtheendresultisaCertificationDefinitionwhichis

savedandassociatedwithaTaskSchedule.Ifapersonisreliedontoperformthistaskinproduction,theprocesscanbepronetoerrorsasusersoftenforgettocheckanoption,includearuleorselectthe

properapplications.Also,usersusuallyselectamanagerhighinthehierarchyandallowthesoftwareto

traversethemanagementchainandgeneratechildcertifications.Thesuggestedscriptingprocessis

goingtodisablethisfunctionalityanddefineuniqueCertificationDefinitionandTaskScheduleobjectsfor

everycertifier.Thiswillprovidetheflexibilitytobreakupthecertificationgenerationprocessintomany

atomicsubtasksallowingforparallelexecutionandfinegrainerrorrecovery.

AtemplateCertificationDefinitionwillbecreatedwhichwillserveasabaseforallcertificationsofthe

sametype.Tocreatethetemplate,simplycreateamanagercertificationintheUIbutintheadvanced

screencheckthe“Forthespecifiedmanagersonly”radiobutton.Thiswillpreventthesoftwarefromtraversingthemanagementchainandgeneratingchildcertifications.

Page 4: Certification Generation - Large Deployments

7/15/2019 Certification Generation - Large Deployments

http://slidepdf.com/reader/full/certification-generation-large-deployments 4/11

BestPractices

Page 4 of 11

Whenalloptionsaresetcorrectly,savetheconfigurationwithanamethatindicatesit’satemplate.

Scheduleitsoitwillneverexecutebysettingthescheduletoatimefarintothefuture.Thiswillbethe

template.Gotothedebugpage( http://hostname:port/iiq/debug )andlocatetheCertificationDefinition

 justcreated.InspecttheoptionsintheXMLandconfirmthatthesettingsarecorrect.

DefineaCertificationManagerTask

CustomTaskDefinition

Thereareanumberofwaystoinputwhichmanagersaregoingtocertifybutforthisexamplea

TaskDefintionwillbedefinedthatwilltakeapopulationasaparameter.Thisisasimpleapproachthat

servesasanexample.Itisprobable,moresophisticatedinputmechanismswillbeneeded.The

populationshouldbeasetofmanagersthathavebeendividedbycriteriathatfulfillsthedesired

behaviorofthecertificationgenerationtask.Ifthecustomerhasasmallinfrastructureitmightbewise

tomakethepopulationalistofmanagersinterleavedbygeographicallocation.Ifgenerationruntimeperformanceisthegoal,thenasimplesplitofthetotalmanagerpopulationdividedbyhowmanytask

executorthreadswouldworknicely.Theexerciseofdeterminingthepopulationsofmanagersto

provideasinputislefttothereader,butcommongroupingsarecoveredinalatersection.

ExampleTaskDefintion

<?xml version='1.0' encoding='UTF-8'?>

<!DOCTYPE TaskDefinition PUBLIC "sailpoint.dtd" "sailpoint.dtd">

<TaskDefinition executor="sailpoint.services.task.GenerateCerts"name="Data Load" resultAction="Delete" type="Generic">

<Description>Schedule certs for population</Description>

<Signature>

<Inputs>

<Argument name="population" required="true" type="string">

<Prompt>Input the population of managers to generate

certifications for.</Prompt>

</Argument>

</Inputs>

<Returns>

<Argument name="result" type="string"><Prompt>Result</Prompt>

</Argument>

</Returns>

</Signature>

</TaskDefinition>

Page 5: Certification Generation - Large Deployments

7/15/2019 Certification Generation - Large Deployments

http://slidepdf.com/reader/full/certification-generation-large-deployments 5/11

BestPractices

Page 5 of 11

DefinetheGenerateCertsTaskExecutor

Thistaskexecutorwillberesponsiblefortakingthelistofmanagerstocertifyandgenerating

CertificationDefintions,TaskSchedulesandlaunchingthem.Itwillalsologitsactivitysotheprocesscan

beresumedintheeventofafailure.

Thegeneralflowoftheprogramisasfollows:

•  Parsethelistofmanagersandverifytheyareactuallymanagers.

•  ForeverymanagerinthelistconstructacloneoftheCertificationDefintiontemplatereferredto

intheearliersection.

•  Setcustomparametersontheclone

o  Changethenametoauniquevalue.Inthisexamplethenameofthemanageris

concatenatedwiththetemplatename.

o  Enforcethe“certificationType”isequaltothedesiredtype.Inthiscase“Manager”.

o  Enforcethe“subordinateCertificationEnabled”attributeissetto“false”toensureonly

certificationsforthespecifiedmanageraregenerated.o  Setthe“certifier”attributetothenameofthemanagerinthecurrentiteration.

•  SavetheCertificationDefintionobjectinthecontextandcommit.

•  CreateanewTaskSchedule

o  SetthenametobeequaltotheCertificationDefintionclonename.

o  Setthelauncherto“spadmin”

o  Setthe“executor”argumentto“CertificationManager”

o  Setthe“resultName”argumenttobeequaltotheCertificationDefintionclonename.

•  Outputlogginginformationaboutwhichmanagerisabouttobeprocessed.

•  DefineaTaskManagerinstanceandrunthedefinedschedule.

•  Blockuntilthetaskreturnsortimesout.

•  Parsetasksresultsforerrorsandwarningsandwritetolog.

•  Continue.

AreferenceimplementationoftheGenerateCertsTaskisprovidedintheappendix.

Page 6: Certification Generation - Large Deployments

7/15/2019 Certification Generation - Large Deployments

http://slidepdf.com/reader/full/certification-generation-large-deployments 6/11

BestPractices

Page 6 of 11

LeveragingtheGenerateCertsTaskExecutorNowthatthescriptisdefinedgeneratingmanagercertificationsacrossinstancescanbeperformed.In

addition,thelargestunitoffailureisnowrestrictedtoonemanager.

ScenarioOne:GenerateManagerCertificationswhileControllingUser

LoadontheWebServerCluster

Withthedefaultfunctionalityofutilizingtheserialmethodofgeneratingcertifications,notificationsare

sentoutfortheentiremanagementhierarchyastheprocesstraversesthelistofmanagers.Theusual

resultofthisiseveryonegetsnotificationstologonanddotheircertificationsatonce.Thiscreatesan

unusuallyhighvolumeofuserloadwhentheserverisalreadybusygeneratingnewcertifications.Orin

separateddeployments(UIserversandbatchservers)theUIserverssimplybecomeoverloadedwith

usersloggingontothesystem.User’sfrustrationescalatesasserverresponsetimeandusabilitydrops.

Oftenthissituationinnoticedafterit’stoolateandthebusinessisalreadylogginganddoingwork.

Typically,customersattempttosolvethisbyaddingmorenodestotheUIserverclustertohandlethe

incomingrequests.However,afterthespikepassesthisinfrastructuresitsunderutilized.Analternative

methodforlargecertificationrunsistophasecertificationsbysomeparameterthatprovidesnatural

loadcontrol.Forexample,ifthecustomer’sorganizationisglobal,thepopulationofmanagerscanbe

interleavedbygeographiclocation.Thatwayallthemanagerswillnotloginanddoworkatthesame

timebecausewhileonegroupisworkingtheotherissleeping.Afterthefirstpartitionofcertificationsis

generated,serverloadismonitored.Thereshouldbeapeakrightwhencertificationsnoticesare

deliveredandactedupon.Thedurationvariesdependentontheorganization’suniquecharacteristics.

Afterthispeakdiesdown,launchthenextpartitionofinterleavedmanagerpopulation.Thenumberof

phasesorpartitionsshouldbesubdividedbyasmanyunitsasthetotalcertificationwindowallows.

ScenarioTwo:CustomerDesiresFasterCertificationGeneration

Performance

BecausetheTaskExecutorreliesonaseparateCertificationDefintionforeachandeverymanager,there

arenodependencesonanypastunitofworkforafutureunitofwork.Ifthecustomerhasarobust

serverinfrastructurethecertificationscanbegeneratedinparallel.Thistypeofsetupisalsodesirablein

testscenarioswheretheentirecertificationrunneedstobetestedinshortertimeframes.To

accomplishthissimplydefineapartitionsizeusingthenumberofavailabletaskexecutionthreadsand

thetotalnumberofmanagers.Bydefaulteach“batchserver”instanceofSailpointcanexecutefive

threadsconcurrently.IftheserverhasalargenumberofphysicalCPUcoresthisnumbercanbe

adjustedtomatchinfrastructure.

Page 7: Certification Generation - Large Deployments

7/15/2019 Certification Generation - Large Deployments

http://slidepdf.com/reader/full/certification-generation-large-deployments 7/11

BestPractices

Page 7 of 11

ToeditthenumberofconcurrentQuartzschedulerthreadsexpandtheidentityiq.jarfoundintheWEB-

INF/libdirectoryandcopythefileiiqBeans.xmltotheWEB-INF/classesdirectory.Thisfilewillnow

overridetheprovidedfile.Openthefileandlookforthefollowinglines.

<bean id="scheduler"

class="org.springframework.scheduling.quartz.SchedulerFactoryBean">

<!-- this avoids the need for a quartz.properties file -->

<property name="quartzProperties">

<props>

<prop

key="org.quartz.threadPool.class">org.quartz.simpl.SimpleThreadPool</prop>

<prop key="org.quartz.threadPool.threadCount">5</prop>

<prop key="org.quartz.threadPool.threadPriority">5</prop>

Changethe“org.quartz.threadPool.threadCount”tothedesiredamountofthreads.Simplymakingthis

numberbiggermay,ormaynot,providebenefit.Tobeonthesafesideitshouldnotbemorethan

doublethenumberofphysicalCPUcoresontheactualserver.Nowtakethatnumberandmultiplyby

howmanybatchserverordefaultservernodesthatareavailable.Thepartitioncannowbecalculated

bydividingthetotalnumberofmanagersbytheclusterwidethreadpoolsize.

Ex.50,000managers/40threadsavailable=apartitionsizeof1,250.

Iftheimplementationisusingapopulationasaninputparameterlikeintheexample-eachpopulation

shouldhaveasizeof1,250users.Thetaskmaylaunchall40instancesofGenerateCertsexecutoratthe

sametime.Optionally,theGenerateCertstaskcanbescriptedtohandlemultiplepopulations.

Substantialperformancegainswillresultfromusingthismethod.

ScenarioThree:TheCertificationGenerationTaskFailsinProduction

Whilethisisnotauniquescenarioaserrorrecoveryshouldbeincorporatedintoscenario1and2,ithas

beenseparatedforclarity.

Duringthegenerationofthecertificationbaddataisencountered,theservergoesdownorsome

unexpectedsituationoccursandthegenerationtaskfails.Ifthecustomerreliedonexecutingonelarge

CertificationDefintionfortheentireorganizationthisfailureiscatastrophic.Usersarealreadyloggingon

tothesystemtoperformworkoncertificationsthatweresent.Theimplementationteamdoesnotknowwheretoresumetheprocessasit’snotclearwhereinthemanagementhierarchytheprocesswas

whenitfailed.

UsingtheGenerateCertstaskthistypeoffailurecanbeavoidedbecausethetaskhasseparatedthe

workintosmallerunitsandloggedthesuccessandfailureofeachunit.Theprocesscannowbe

resumedwithnoimpacttothe“successful”population.Thesimplestimplementationofthisisjustto

readthelogafterafailureandtweakthepopulationsandre-executethetask.Amoresophisticated

Page 8: Certification Generation - Large Deployments

7/15/2019 Certification Generation - Large Deployments

http://slidepdf.com/reader/full/certification-generation-large-deployments 8/11

BestPractices

Page 8 of 11

approachistowriteasuccessorfailuremarkertoatableduringexecutionandwhentheGenerateCerts

taskisexecutedagainitcanskipsuccessfulmembersofthepopulation.

Page 9: Certification Generation - Large Deployments

7/15/2019 Certification Generation - Large Deployments

http://slidepdf.com/reader/full/certification-generation-large-deployments 9/11

BestPractices

Page 9 of 11

 Appendix

ExampleimplementationofGenerateCertsTaskExecutor

packagesailpoint.services.task;

mportsailpoint.api.*;

mportsailpoint.object.*;

mportsailpoint.task.AbstractTaskExecutor;

mportsailpoint.tools.Message;

mportsailpoint.tools.xml.XMLObjectFactory;

mportsailpoint.tools.xml.XMLReferenceResolver;

mportjava.util.ArrayList;

mportjava.util.List;

/**

*CreatedbyIntelliJIDEA.

*User:nwellinghoff

*Date:10/27/11

*Time:4:16PM

*/

publicclassGenerateCertsextendsAbstractTaskExecutor{

publicvoidexecute(SailPointContextsailPointContext,TaskScheduletaskSchedule,TaskResulttaskResult,Attributes<String

Object>stringObjectAttributes)throwsException{

StringcsvManagerList="Andrea.Hudson,Randy.Knight";//putyourcommaseparatedlistofmanageridshere.

//Ifyouhavemultiplenodestoexecuteonyoucansplitthislistandexecutemultipleinstancesoftheruleperlist

String[]managerList=csvManagerList.split(",");

SailPointContextcontext=SailPointFactory.getCurrentContext();

//Putnameofyourtemplatedefhere.

CertificationDefinitiontemplateCertDef=context.getObject(CertificationDefinition.class,"ManagerCertificationTemplate

//5.1version

//templateCertDef.setIsSubordinateCertificationEnabled;

//4.0p11version

//templateCertDef.getAttributes().put("subordinateCertificationEnabled",false);

interrors=0;

Page 10: Certification Generation - Large Deployments

7/15/2019 Certification Generation - Large Deployments

http://slidepdf.com/reader/full/certification-generation-large-deployments 10/11

BestPractices

Page 10 of 11

for(inti=0;i<managerList.length;i++){

StringmanagerId=managerList[i];

Identitycertifier=null;

//makeacopyofthetemplate

CertificationDefinitionnewCertDef=(CertificationDefinition)

XMLObjectFactory.getInstance().cloneWithoutId(templateCertDef,(XMLReferenceResolver)context);

//Thisflagwillonlygenerateacertforthemanagerinquestion.

//4.0p11version

newCertDef.getAttributes().put("subordinateCertificationEnabled","false");

try{

System.out.println("################################################################");

certifier=context.getObjectByName(Identity.class,managerId);

if(certifier==null){

System.out.println("CouldnotfindIdentitywithname"+managerId);

break;

}

System.out.println("Generatingcertificationformanager"+certifier.getName());

newCertDef.setName("ManagerCertificationfor"+certifier.getName());

//enforcethatthiscertisamanagercertnotaglobalmanagercert

newCertDef.getAttributes().put("certificationType","Manager");

//4.0p11version;

newCertDef.getAttributes().put("certifier",certifier.getName());

//5.1version;

//newCertDef.setCertifierName(certifier.getName());

System.out.println(newCertDef.getName());

context.saveObject(newCertDef);

context.commitTransaction();

context.attach(newCertDef);

//NowcreateaCertificationScheduleforthisdef.

TaskSchedulets=newTaskSchedule();

ts.setName("ManagerCertificationSchedulefor"+certifier.getName());

ts.setArgument("certificationDefinitionId",newCertDef.getId());

ts.setLauncher("spadmin");

ts.setArgument("executor","CertificationManager");

ts.setArgument("resultName","ManagerCertificationSchedulefor"+certifier.getName());

Page 11: Certification Generation - Large Deployments

7/15/2019 Certification Generation - Large Deployments

http://slidepdf.com/reader/full/certification-generation-large-deployments 11/11

BestPractices

Page 11 of 11

askManagertm=newTaskManager(context);

System.out.println("Launchingcertification...");

System.out.println(newCertDef.toXml());

System.out.println("WithSchedule...");

System.out.println(ts.toXml());

tm.runNow(ts);

//ThiswillreturnwhenthecertrunisfinishedORthetimeoutoccurs.Changethistimeouttothemax

//certgenerationtimeplus20%.Timeisinseconds.

TaskResulttr=tm.awaitTask(ts,1800);

if(tr.getErrors()!=null){

System.out.println("Errorwhilelaunchingcert"+ts.getName());

for(Messagem:tr.getErrors()){

System.out.println("Error"+m.getMessage());

}

}

if(tr.getWarnings()!=null){

System.out.println("Warnwhilelaunchingcert"+ts.getName());

for(Messagem:tr.getWarnings()){

System.out.println("warn"+m.getMessage());

}

}

}catch(Exceptione){

errors++;

System.out.println("FAILURE:Certificationgenerationfailedonid"+certifier.getName()+"withsystemidof"+

ertifier.getId()+".Resumingoperationonlist.");

e.printStackTrace();

}

finally{

System.out.println("Cleaningupgeneratedobjects...");

try{

Terminatorterminator=newTerminator(context);

terminator.deleteObject(newCertDef);

context.decache();

}catch(Exceptione){

e.printStackTrace();

}

System.out.println("################################################################");

}

}

System.out.println("Finishedwith"+errors+"errors.Iferrorcountis>0checkoutputforsrc");

}