granules: a lightweight runtime for scalable computing...
TRANSCRIPT
201
Chapter 9
Granules: A Lightweight Runtime for Scalable Computing with Support for Map-Reduce
Shrideep Pallickara, Jaliya Ekanayake, and Geoffrey Fox
Contents9.1 Introduction.............................................................................................2029.2 NaradaBrokering......................................................................................2039.3 Granules...................................................................................................205
9.3.1 ComputationalTask.....................................................................2059.3.2 DatasetsandCollections...............................................................2069.3.3 SpecifyingaSchedulingStrategy...................................................2069.3.4 Finite-StateMachineforaComputationalTask...........................2079.3.5 InterleavingExecutionofComputationalTasks............................208
9.3.5.1 SizingThread-Pools........................................................2089.3.6 Diagnostics...................................................................................208
K10175_C009.indd 201 3/2/2010 12:52:27 PM
202 ◾ Cloud Computing and Software Services: Theory and Practice
9.1 IntroductionCloud computing has gained significant traction in recent years. By facilitatingaccesstoanelastic(meaningtheavailableresourcepoolthatcanexpandorcon-tractovertime)setofresources,cloudcomputinghasdemonstrableapplicabilitytoawiderangeofproblemsinseveraldomains.
Appealing featureswithin cloud computing include access to a vastnumberof computational resources and inherent resilience to failures.The latter featurearises,because incloudcomputingthefocusofexecutionisnotaspecific,well-knownresourcebutratherthebestavailableone.Anothercharacteristicofalotofprogramsthathavebeenwrittenforcloudcomputingisthattheytendtobestate-less.Thus,when failuresdo takeplace, theappropriatecomputationsare simplyrelaunchedwiththecorrespondingdatasets.
Amongthe forces thathavedriven theneed forcloudcomputingare fallinghardwarecostsandburgeoningdatavolumes.Theabilitytoprocurecheaper,morepowerfulCPUscoupledwithimprovementsinthequalityandcapacityofnetworkshavemadeitpossibletoassembleclustersatincreasinglyattractiveprices.Thepro-liferationofnetworkeddevices,Internetservices,andsimulationshasresultedin
9.4 SupportforMap-ReduceinGranules......................................................2099.4.1 TwoSidesoftheSameCoin.........................................................2109.4.2 SettingUpGraphs........................................................................ 2119.4.3 CreatingComputationalPipelines................................................ 2119.4.4 ObservingtheLifeCycleofaPipeline..........................................212
9.5 DevelopingandDeployingApplicationsUsingGranules.........................2129.5.1 DevelopingApplications...............................................................212
9.5.1.1 Initialization...................................................................2139.5.1.2 ProcessingLogic.............................................................2139.5.1.3 SchedulingStrategy........................................................213
9.5.2 DeployingApplicationsUsingGranules.......................................2149.5.2.1 InitializingCommunicationsandResourceDiscovery.....2149.5.2.2 InitializingandDeployingComputationalTasks...........2149.5.2.3 Tracking/SteeringaDeployedApplication..................... 215
9.6 RelatedWork........................................................................................... 2159.7 Benchmarks.............................................................................................217
9.7.1 StreamingSubstrate......................................................................2179.7.2 InformationRetrieval:Exactly-Once............................................2189.7.3 K-Means:Iterative.........................................................................2199.7.4 PeriodicScheduling......................................................................2209.7.5 DataDriven..................................................................................2219.7.6 AssemblingmRNASequences.......................................................222
9.8 Conclusions..............................................................................................224References.........................................................................................................224
K10175_C009.indd 202 3/2/2010 12:52:27 PM
Granules ◾ 203
largevolumesofdatabeingproduced.This,inturn,hasfueledtheneedtoprocessandstorevastamountsofdata.Thesedatavolumescannotbeprocessedbyasinglecomputerorasmallclusterofcomputers.Furthermore,inmostcases,thisdatacanbeprocessedinapleasinglyparallelfashion.Theresulthasbeentheaggregationofalargenumberofcommodityhardwarecomponentsinvastdatacenters.
Map-Reduce[1], introducedbyDeanandGhemawatatGoogle, is themostdominantprogrammingmodelfordevelopingapplicationsincloudsettings.Here,largedatasetsaresplitintosmaller,moremanageablesizes,whicharethenprocessedbymultiplemapinstances.Theresultsproducedbyindividualmapfunctionsarethensenttoreducers,whichcollatethesepartialresultstoproducethefinaloutput.A clear benefit of such concurrent processing is a speed-up that is proportionaltothenumberofcomputationalresources.Map-ReducecanbethoughtofasaninstanceoftheSPMD[2]programmingmodelforparallelcomputingintroducedbyFedericaDarema.ApplicationsthatcanbenefitfromMap-Reduceincludedataand/ortask-parallelalgorithmsindomainssuchasinformationretrieval,machinelearning,graphtheory,andvisualization,amongothers.
In this chapter,which is an extendedversionofourpaper [21],wedescribeGranules [3], a lightweight streaming-based runtime for cloud computing.Granulesallowsprocessing tasks tobedeployedona single resourceora setofresources.BesidesthebasicsupportforMap-Reduce,wehaveincorporatedsup-portforvariantsoftheMap-Reduceframeworkthatareparticularlysuitableforscientificapplications.UnlikemostMap-Reduceimplementations,Granulesusesstreaming fordisseminating intermediate results, asopposed tousingfile-basedcommunications.Thisleadstodemonstrablybetterperformance(seebenchmarksinSection9.7).
Thischapterisorganizedasfollows.InSection9.2,weprovideabriefoverviewoftheNaradaBrokeringsubstratethatweuseforstreaming.WediscusssomeofthecoreelementsofGranulesinSection9.3.Section9.4outlinesoursupportforMap-Reduceandforthecreationofcomplexcomputationalpipelines.Section9.5describestheprocessofdevelopinganddeployingapplicationsusingGranules.InSection9.6,wedescriberelatedworkinthisarea.InSection9.7,weprofileseveralaspectsoftheGranulesruntime,andwherepossible,contrastitsperformancewithcomparablesystems,suchasHadoop,Dryad,andMPI(MessagePassingInterface).InSection9.8,wepresentourconclusions.
9.2 NaradaBrokeringGranulesuses theNaradaBrokering[4–6]streamingsubstrate(developedbyus)forallitsstreamsdisseminations.TheNaradaBrokeringcontentdistributionnet-work(depictedinFigure9.1)comprisesasetofcooperatingrouternodesknownasbrokers.Producersandconsumersdonotdirectlyinteractwitheachother.Entities,whichareconnected tooneof thebrokerswithin thebrokernetwork,use their
AQ2
AQ3
AQ4
AQ5
AQ1
K10175_C009.indd 203 3/2/2010 12:52:27 PM
204 ◾ Cloud Computing and Software Services: Theory and Practice
hostingbroker to funnel streams into thebrokernetworkand, fromthereon, tootherregisteredconsumersofthosestreams.
NaradaBrokeringisapplicationindependentandincorporatesseveralservicestomitigatenetwork-inducedproblemsasstreamstraversedomainsduringdissemi-nations.Thissystemprovisionseasy-to-useguaranteeswhiledeliveringconsistentandpredictableperformancethatisadequateforuseinreal-timesettings.
Consumersofagivendatastreamcanspecify,veryprecisely,theportionsofthedatastreamthattheyareinterestedinconsuming.Bypreferentiallydeployinglinksduringdisseminations,theroutingalgorithm[4]inNaradaBrokeringensuresthattheunderlyingnetworkisoptimallyutilized.Thispreferentialroutingensuresthatconsumersreceiveonlythoseportionsofstreamsthatareofinteresttothem.Sinceagivenconsumeristypicallyinterestedinonlyafractionofthestreamspresentinthesystem,preferentialroutingensuresthataconsumerisnotdelugedbystreamsthatitwillsubsequentlydiscard.
Thesystemincorporatessupportforreliablestreamingandsecurestreaming.Inreliablestreaming,thesubstratecopeswithdisconnectsandprocess/linkfailuresofdifferentcomponentswithinthesystemwiththeabilitytofine-tuneredundancies[5]foraspecificstream.Securestreaming[6]enforcestheauthorizationandcon-fidentialityconstraintsassociatedwiththegenerationandconsumptionofsecurestreamswhilecopingwithdenial-of-serviceattacks.
Some of the domains that NaradaBrokering has been deployed in includeearthquakescience,particlephysics,environmentalmonitoring,geosciences,GISsystems,anddefenseapplications.AQ6
Producers or consumers
Broker node
Figure 9.1 NaradaBrokering broker network.
K10175_C009.indd 204 3/2/2010 12:52:29 PM
Granules ◾ 205
9.3 GranulesGranulesorchestratestheconcurrentexecutionofprocessingtasksonadistributedsetofmachines.Granules is itselfdistributed,and its componentspermeatenotonlythecomputationalresourcesonwhichitinterleavesprocessing,butalsothedesktopfromwheretheapplicationsarebeingdeployedinthefirstplace.Therun-timemanagestheexecutionofasetofprocessingtasksthroughvariousstagesoftheirlifecycle:deployment,initialization,execution,andtermination.Figure9.2depictsthevariouscomponentsthatcompriseGranules.
9.3.1 Computational TaskThemostfundamentalunitinGranulesisthenotionofacomputational task.Thiscomputational taskencapsulatesprocessing functionality, specifies its schedulingstrategy, and operates on different types of datasets. These computational taskscantakeonadditionalinterchangeableroles(suchasmapandreduce)and,whencascaded,canformcomplexexecutionpipelines.
Computationaltasksrequirethedomainspecialiststospecifyprocessingfunc-tionality.Thisprocessingtypicallyoperatesuponacollectionofdatasetsencapsu-latedwithinthecomputationaltask.
Thecomputational task encapsulates functionality forprocessing for a givenfine-grainedunitofdata.Thisdatagranularitycouldbeapacket,afile,asetoffiles,oradatabaserecord.Forexample,acomputationaltaskcanbewrittentoevaluatearegularexpressionquery(grep)onasetofcharacters,afile,orasetoffiles.Insome
R1
GR2
GRN
G
C CG
AG
Discovery services
Life-cycle management
Diagnostics
Deployment manager
Concurrent execution
queueDataset
initializer
Content distribution network
A A
GranulesGC Client Computational resourcesRX
Granules
Figure 9.2 Overview of the Granules runtime.
K10175_C009.indd 205 3/2/2010 12:52:29 PM
206 ◾ Cloud Computing and Software Services: Theory and Practice
cases,therewillnotbeaspecificdataset;rather,eachcomputationaltaskinstanceinitializesitselfusingarandom-seedgenerator.
Computationaltasksincludeseveralmetadata,suchasversioninginformation,timestamps,domainidentifiers,andcomputationidentifiers.Individualinstancesofthecomputationaltasksincludeinstanceidentifiersandtaskidentifiers,whichinturnallowsustogroupseveralrelatedcomputationaltaskstogether.
9.3.2 Datasets and CollectionsIn Granules, datasets are used to simplify access to the underlying data type.Datasetscurrently supportedwithinGranules include streamsandfiles; supportfordatabasesisbeingincorporated.Foragivendatatype,besidesmanagingtheallocationandreclamationofassortedresources,Granulesalsomediatesaccesstoit.Forexample,Granulesperformsactionsrelatedtosimplifyingtheproductionandconsumptionofstreams,readingandwritingoffiles,andtransactionalaccesstodatabases.
Adatacollectionisassociatedwitheverycomputationaltask.Adatacollectionrepresentsacollectionofdatasets,andmaintainsinformationaboutthetype,num-ber,andidentifiersassociatedwitheveryencapsulateddataset.
Allthatthedomainspecialistneedstospecifyisthenumberandtypeofdatas-etsinvolved.Thesystemimposesnolimitsonthenumberofdatasetswithinadata-set collection.During initializationsof thedataset collection,dependingon thetypeoftheconstituentdatasets,Granulessubscribestotherelevantstreams,con-figuresaccesstofilesonnetworkedfilesystems,andsetsupconnections(JDBC)tothedatabases.
Datasetcollectionsallowobservers tobe registered to trackdataavailability,datasetinitializations,andclosure.Thissimplifiesdataprocessing,sinceitobviatestheneedtoperformpolling.
9.3.3 Specifying a Scheduling StrategyComputational tasks specify a scheduling strategy, which in turn governs theirlifetimes.Computationaltaskscanspecifytheirsched-ulingstrategyalongthreedimensions(seeFigure9.3).Thecountsaxisspecifiesthenumberoftimesacompu-tationaltaskneedstobeexecuted.Thedata drivenaxisspecifiesthatthecomputationaltaskneedstobesched-uled for execution whenever data is available on anyoneofitsconstituentdatasets.Theperiodicityaxisspec-ifiesthatcomputationaltasksbeperiodicallyscheduledforexecutionatpredefinedintervals(specifiedinms).
Eachoftheseaxescanextendtoinfinity,inwhichcase, it constitutes a stay-alive primitive. A domain
AQ7
Counts
Data driven
Periodicity
Figure 9.3 Dimensions for scheduling strategy.
K10175_C009.indd 206 3/2/2010 12:52:30 PM
Granules ◾ 207
specialistcanalsospecifyacustomschedulingstrategythatpermutesalongthesethreedimensions.Thus,onecanspecifyaschedulingstrategythatlimitsacompu-tationaltasktobeexecutedamaximumof500timeseitherwhendataisavailableoratregularintervals.
Acomputationaltaskcanchangeitsschedulingstrategyduringexecution,andGranuleswill enforce thenewly established scheduling strategyduring thenextround of execution (Section 9.3.5). This scheduling change can be a significantone—fromdatadriventoperiodic.Theschedulingchangecouldalsobeaminoronewithchangestothenumberoftimesthecomputationneedstobeexecuted,orwithanupdatetotheperiodicityinterval.
In addition to the aforementioned primitives, another primitive—stay alive until termination condition reached—canbespecified.Inthiscase,thecomputa-tionaltaskcontinuestobe“stayalive”untilthecomputationaltaskassertsthatitsterminationconditionhasbeenreached.Theterminationconditionoverridesanyotherprimitivesthatmayhavebeenspecifiedandresultsinthegarbagecollectionofthecomputationaltask.
9.3.4 Finite-State Machine for a Computational TaskAt a given computational resource, Granules maintains a finite-state machine(FSM)foreverycomputationaltask.ThisFSM,depictedinFigure9.4,hasfourstates:initialize,activated,dormant,andterminate.
The transition triggers for this FSM include external requests, elapsed timeintervals,dataavailability,resetcounters,andassertionsoftheterminationcondi-tionbeingreached.
Whenacomputationaltaskisfirstreceivedinadeploymentrequest,Granulesproceedstoinitializethecomputationaltask.TheFSMcreatedforthiscomputa-tionaltaskstartsoffintheinitializestate.
If, for some reason, the computational task cannotproceed in its execution,eitherbecausethedatasetsarenotavailableorthestart-uptimehasnotyetelapsed,thecomputationaltasktransitionsintothedormantstate.Iftherewereproblemsininitialization,thecomputationaltasktransitionsintotheterminatestate.
Terminate
Initialize
Dormant
Activated
Figure 9.4 FSM for a computational task.
K10175_C009.indd 207 3/2/2010 12:52:30 PM
208 ◾ Cloud Computing and Software Services: Theory and Practice
If,ontheotherhand,thecomputationaltaskwasinitializedsuccessfully,andisreadyforexecutionwithaccessibledatasets,ittransitionsintotheactivatedstate.
9.3.5 Interleaving Execution of Computational TasksAteachcomputationalresource,Granulesmaintainsapoolofworkerthreadstomanageandinterleavetheconcurrentexecutionofmultiplecomputationaltasks.
Whenacomputational task isactivatedandreadyforexecution, it ismovedintotheactivatedqueue.Asandwhenworkerthreadsbecomeavailable,thecom-putationaltasksarepulledfromtheFIFOqueueandexecutedinaseparatethread.Uponcompletionofthecomputationaltask,theworkerthreadisreturnedbacktothethread-pool,tobeusedtoexecuteotherpendingcomputationaltaskswithintheactivatedqueue.ThecomputationaltaskisplacedeitherinthedormantqueueorscheduledforgarbagecollectiondependingonthestateofitsFSM.
Afteracomputationaltaskhasfinisheditslatest(orthefirst)roundofexecu-tion,checksaremadetoseeifitshouldbeterminated.Todoso,theschedulingstrategy associated with the computational task is retrieved. If a computationaltaskneedstoexecuteafixednumberoftimes,acheckismadetoseeifthecounterhasreset.Ifthecomputationaltaskspecifiesastay-aliveprimitivebasedeitherondataavailabilityorperiodicity,checksaremadetoseeifthedatasetscontinuetobeavailableoriftheperiodicityintervalhaselapsed.Acheckisalsomadetoseeifthecomputationaltaskhasassertedthatitsterminationconditionhasbeenreached.
Ifnoneofthesechecksindicatethatthecomputationaltaskshouldbetermi-nated,itisscheduledforanotherroundofexecutionorittransitionsintothedor-mantstate.Acomputationaltaskcancontinuallytogglebetweenthedormantandtheactivatedstatetillaterminationconditionhasbeenreached.
9.3.5.1 Sizing Thread-Pools
Thenumberofworkerthreadswithinthethread-poolisconfigurable.Ingeneral,thenumberofthreadsneedstobebalancedsothattheaccruedconcurrencygainsarenotoffsetbycontext-switchingoverheadsamongthethreads.Asageneralrule,it isagoodideatoset thisnumbertobeapproximatelyequal tothenumberofexecutionpipelinesavailableonagivenmachine.Thus,foraquad-coreCPUwithtwoexecutionpipelinespercore,thethread-poolwillbesetuptohaveapproxi-matelyeightthreads.
9.3.6 DiagnosticsInGranules,ausercantrackthestatusofaspecificcomputationaltaskorcollec-tions(job)ofcomputationaltasks.Thesystemmaintainsdiagnostic informationabouteverycomputational task.This includes informationabout thenumberoftimesacomputationaltaskwasscheduledforexecution,itsqueuingoverheads,its
AQ8
K10175_C009.indd 208 3/2/2010 12:52:31 PM
Granules ◾ 209
CPU-boundtime,thetimeitwasmemory-resident,andthetotalexecutiontime.Acomputationaltaskcanalsoassertthatdiagnosticmessagesbesentbacktothecli-entduringany(orsome)ofitsstatetransitions.Ontheclientside,anobservercanberegisteredforcollectionsofcomputationaltaskstotracktheirprogresswithouttheneedtoactivelypollindividualcomputationaltasks.
9.4 Support for Map-Reduce in GranulesMap-Reduce is the dominant framework used in cloud computing settings. InMap-Reduce,alargedatasetisbrokenupintosmallerchunksthatareconcurrentlyoperateduponbymapfunctioninstances.Theresultsfromthesemapfunctions(usually,<key,value>pairs)arecombinedinthereducers,whichcollatethevaluesfor individualkeys.Typically, therearemultiple reducers, and theoutputs fromthesereducersconstitutethefinalresult.ThisisdepictedinFigure9.5.
TheMap-Reduceframeworkhasseveraladvantages.First,thedomainscientistonlyneedstoprovidetheMap-Reducefunctionalityandthedatasets.Second,itistheresponsibilityoftheframeworktotransparentlyscaleasthenumberofavailableresources,andtheproblemsize,increases.Finally,theorchestrationoftheconcur-rentdata-parallelexecutionismanagedbytheframework.
IntraditionalMap-Reduce,intermediatestagesexchangeresultsusingasetof<key,value>pairs.Wehaveincorporatedsupportforthisbasicresulttype.Butwehavealsoincorporatedsupportforexchangeofprimitivedatatypes,suchasint,short,boolean,char,long,float,anddouble.Wehavealsoincorpo-ratedsupportforexchangingarrays([])and2Darrays([][])oftheseprimitivedatatypes.ThereisalsosupportforexchangingObjectsthatencapsulatedcompounddatatypes,alongwitharraysand2DarraysoftheseObjects.
TheintermediateresultsinmostMap-ReduceimplementationsutilizefileIOfor managing results produced by the intermediate stages. The framework thennotifiesappropriatereducerstopullorretrievetheseresultsforfurtherprocessing.
d1
d2
dN
Map1
Map2
MapN
Reduce1
ReduceK
o1
oK
Dataset
Figure 9.5 Basic Map-Reduce framework.
K10175_C009.indd 209 3/2/2010 12:52:31 PM
210 ◾ Cloud Computing and Software Services: Theory and Practice
Dependingontheapplication,theoverheadsintroducedbyperformingsuchdisk-IO can be quite high. In Granules, we use streaming to push these resultsontoappropriatereducers.Streaming,asvalidatedbyourbenchmarks(describedinSection9.7),issignificantlyfaster,andwethinkthatthereareseveralclassesofapplicationsthatcanbenefitfromthis.
Additionally,sincetheresultsarebeingstreamedasandwhentheyhavebeencomputed, successive stages have access to partial results from preceding stagesinsteadofwaitingfortheentirecomputationtocomplete.Thisisparticularlyuse-fulinsituationswhereoneisinterestedingettingasmanyresultsaspossiblewithinafixedamountoftime.
9.4.1 Two Sides of the Same CoinInGranules,mapandreducearetworolesassociatedwiththecomputationaltask.Theserolesinheritallthecomputationaltaskfunctionality,whileaddingfunction-alityspecifictotheirroles.
The map role adds functionality related to adding, removing, tracking, andenumeratingthereducersassociatedwiththemapfunction.Typically,amapfunc-tionhasonereducerassociatedwithit.InGranules,wedonotlimitthenumberofreducersassociatedwithamapfunction.Thisfeaturecanbeusedtofine-tuneredundancieswithinacomputationalpipeline.
Thereduceroleaddsfunctionalityrelatedtoadding,removing,tracking,andenumeratingmapsassociatedwith it.Thereducerhas facilities totrackoutputgeneratedbytheconstituentmaps.Specifically,areducercandetermineifpar-tialorcompleteoutputshavebeenreceivedfromthemaps.Thereducerolealsoincorporates support to detect and discard any duplicate outputs that may bereceived.
Themapand reduce roleshave facilities to create andpublish results.Thepayloadsfortheseresultscanbeprimitivedatatypesthatwediscussedearlier,Objects encapsulatingcompounddata types,<key,value>pairs,arrays,and2Darraysofthesame.InGranules,generatedresultsincludesequencinginfor-mationandmetadataspecifictothegenerator.Additionally,anentityisallowedto assert if these results are partial results and/or if the processing has beencompleted.
SincemapandreducearetworolesofthecomputationaltaskinGranules,theyinherit functionality related to scheduling strategy (and life-cyclemanagement),diagnosticstrategy,anddatasetmanagement.
Individual map and reduce instances toggle between the activated and dor-mant states (Section9.3.5) till such time that theyare ready toassert that theirterminationconditionhasbeenreached.Forexample,areducermayassertthatithasreacheditsterminationconditiononlyafterithasreceived,andprocessed,theoutputsofitsconstituentmaps.
K10175_C009.indd 210 3/2/2010 12:52:31 PM
Granules ◾ 211
9.4.2 Setting Up GraphsGranulessupportsasetofoperationsthatallowgraphstobesetup.Individualmaps can add/remove reducers. Similarly, reducers are allowed to add/removemaps. The functions are functionally equivalent. Granules also allows the mapandreducerolestobeinterchangable:amapcanactasareducer,andviceversa.Figure9.6depictshowsupportforaddition/removalofrolescombinedwithroleinterchangeability can be used to create a graph with a feedback loop. In ourbenchmarks, involving thek-meansmachine learning algorithm,wehave threestageswithafeedbackloopfromtheoutputofstage2toitsinput.Granulesman-agesoverheadsrelatedtoensuringthattheoutputsfromthemapareroutedtothecorrectreducers.
Additionally,Granulescancreateexecutiongraphsoncethenumbersofmapandreduceinstancesinapipelinehavebeenspecified.Granulesensurestheappro-priatelinkageoftheMap-Reduceinstances.
9.4.3 Creating Computational PipelinesTypically, inMap-Reduce,theinstancesthatcompriseanexecutionpipelineareorganized in a directed acyclic graph (DAG), with the execution proceeding insequencethroughmonotonicallyincreasingstages.
InGranules,wehaveincorporatedsupportforcyclestobepresent.ThisallowsGranulestofeedbacktheoutputsofsomestage,withinapipeline,toanyofitspre-cedingstages.Thesystemplacesnorestrictionsonthespanlength,orthenumber,ofthefeedbackinthepipeline.InasenseitcanbearguedthatGranulessupportsbothdata-andcontrol-flowgraphs.AnexampleofsuchacomputationalgraphinGranulesisdepictedinFigure9.7.
One feature of the computational task plays a role in allowing these loops:the notion of the stay-alive computation. Furthermore, since this is available atthemicrolevel(computationaltask),individualstages,collectionofstages,orthe
AQ9
M1
M2
M3
M4
R1
R2
RMRM.addreduce (M1)RM.addreduce (M2)RM.addreduce (M3)RM.addreduce (M4)
Figure 9.6 Creating a simple feedback loop.
K10175_C009.indd 211 3/2/2010 12:52:32 PM
212 ◾ Cloud Computing and Software Services: Theory and Practice
computationalpipelineitselfcanbedependentoniterative,periodic,data-driven,orterminationconditions.
Granulesmanagesthepipelinecomplexity.Thedomainscientistdoesnotneedtocopewithfan-incomplexity,whichcorrespondstothenumberofunitsthatfeedresultsintoagiveninstance.Onceapipelinehasbeencreated,adomainspecialistdoesnothavetocopewithIO,synchronization,ornetworking-relatedissues.Theruntimeincludesfacilitiestotrackoutputsfromprecedingstages.
9.4.4 Observing the Life Cycle of a PipelineAt the client side, during the deployment process, Granules allows a life-cycleobserver toberegistered foranexecutionpipeline.Thisobserverprocessesdiag-nosticmessagesreceivedfromdifferentcomputationalresourcesrunningGranules.Thesediagnosticmessagesrelate tostate transitionsassociatedwiththedifferentcomputational task instances (and the map and reduce roles) and the pertinentmetricsassociatedwiththecomputationtask.Thelife-cycleobserverreportstotheclientuponcompletionofanexecutionpipeline.Theobserveralsoreportserrorsintheexecutionofanyoftheunitsthatcomprisethepipeline.
9.5 Developing and Deploying Applications Using Granules
Inthissection,wedescribetheprocessofdevelopinganddeployingapplicationsusingGranules. Inboth cases,Granules incorporates support forutility classes,whosebehaviormaybeextendedtosuitspecificneeds.
9.5.1 Developing ApplicationsGranules simplifies the process of developing applications. Developers sim-ply extend theMapReduceBase class.This class implements functionality thatencompassesboththemapandreducerolesofacomputation.Onerequirementisthatthederivedclasshasexactlyoneconstructor,whichdoesnottakeanyargu-ments.Developersof thederived classonlyneed to implement theexecute()
Stage 1 Stage 2 Stage 3
Figure 9.7 Creating pipelines with cycles.
K10175_C009.indd 212 3/2/2010 12:52:33 PM
Granules ◾ 213
method.Typicalstepsinvolvedinimplementingthismethodincludeinitializationofthedatasetsanddatastructures,processinglogic,andspecificationofaschedul-ingstrategy.
9.5.1.1 Initialization
Typically, depending on the type of the dataset, initialization of the datas-ets involved in the processing is performed automatically. The designer sim-plyspecifiestheidentifiersforthedataset.Initializationsofthedatastructuresneededbythecomputationcanbeperformedeitherinthenullconstructororintheexecute()method.Inthelattercase,caremustbetakentoensurethattheinitializationsareperformedonlyonceacrosssuccessiveinvocationsoftheexecute()method.
9.5.1.2 Processing Logic
Theprocessinglogicwithintheexecute()methodisdomainspecific.Thispro-cessingwouldinvolveeitherthegenerationofresults,orthemanagementandcol-lationofpreviouslyproducedresults.Inthereducerole,itisalsopossibletocheckifoutputshavebeenreceivedfromalltheprecedingmapsinadditiontodiscardinganyduplicateresultsthatweregenerated.
Thegenerationofresultsiseasy,andthesystemallowsentitiestoattachdiffer-entpayloadstotheseresults.Thesystemcurrentlyallowsforthepayloadsfortheseresultstobe<key,value>pairs,wheretheelementsofthesetuplescouldbeobjectsthatencapsulatecompounddatatypes.Thesystemallowsinstances,arrays([]),and2Darrays ([][]) of primitive data types such asint,short,long,double,float,andchartobeattachedaspayloadsoftheseresults.Thesystemhandlesthemarshallingandun-marshallingofthesepayloadsautomatically.
Theprocessing logic alsoneeds to copewith exceptions thatwillbe thrownasresultsoftheprocessing.Theseexceptionscouldresultfromproblemswiththedatasets,marshallingissues,andnetworkingproblems.
9.5.1.3 Scheduling Strategy
Acomputational task can change its scheduling strategyduring execution.Thischangeisreflectedduringthenextiterationoftheexecute()method.Thesys-temenforcesthenewlycreatedschedulingstrategyassoonasthecurrentiterationoftheexecute()methodterminates.Computationaltasksthathavespecifiedaschedulingstrategythatconstituteseitherastay-aliveprimitive,orimpliesacertainnumberofiterations,canassertthattheirterminationconditionhasbeenreached.Atthistime,thecomputationaltaskisscheduledforgarbagecollectionassoonascontrolreturnsfromtheexecute()method.
K10175_C009.indd 213 3/2/2010 12:52:33 PM
214 ◾ Cloud Computing and Software Services: Theory and Practice
9.5.2 Deploying Applications Using GranulesGranules provides a helper class, the InstanceDeployer, to enable applications,andthecomputationaltasksthatcompriseit,tobedeployedonasetofresources.This class performs several operations related to initializing communications,resource discovery, and deployment of computations. It is recommended that adeployerbecreatedforeachapplication.ThiscanbedonebysimplyextendingtheInstanceDeployer.
9.5.2.1 Initializing Communications and Resource Discovery
Thefirst stepthatanapplicationdeployerneeds toperformis to initializecom-municationswiththecontentdistributionnetwork(NaradaBrokering).Thiscanbeperformedbyinvokingtheconstructorforthebaseclass(InstanceDeployer),whichtakesasetofpropertiesasitsargument.Thisistypicallydonebyinvokingthesuper(streamingProperties)inthederivedclass’sconstructor.Someoftheelementsthataretypicallypartofthissetofpropertiesincludethehostname,theport, and the transport type foroneof the routernodeswithin the contentdisseminationnetwork.Dependingonthetransportoverwhichcommunicationstakeplace,therewouldbeadditionalelementsthatmayneedtobespecified.Forexample,iftheSSLcommunicationsareused,additionalelementsthatneedtobespecifiedincludethelocationsofthetruststoreandthekeystorethatwouldbeusedforsecurecommunications.
Oncecommunicationshavebeenestablished,Granulesautomaticallydiscov-ersresourcesthatarecurrentlyavailable.Thislistcouldbeperiodicallyrefreshedshouldtheneedarise.
9.5.2.2 Initializing and Deploying Computational Tasks
Thedeveloperthenneedstoprovideamethodthatinitializesthecomputationaltasks.Thisinvolvesoneormoreofthefollowing:
1.Initializing the Processing Directives associated with an instance:Thesedirec-tivesareusedtoencodeinstance-specificinformationthatisaccessibleonlytotheinstanceinquestion.
2.Specification of the datasets and collection associated with the computation:Granulesisresponsibleforconfiguringaccesstothesedatasets.
3.Linking of the Map-Reduce roles:Granules ensures that once-linked resultsproducedbythemapsareautomaticallyroutedtotheappropriatereducers.
4.Specifying the scheduling strategy for the computational tasks:Bydefault, theexactly-onceschedulingstrategyisused.
5.Distribution of datasets across these instances:Granules incorporatesutilitiesthatallowthisdistributiontobeperformedefficiently.
AQ10
K10175_C009.indd 214 3/2/2010 12:52:33 PM
Granules ◾ 215
Todeployanapplication,thedeveloperonlyneedstoinvokethedeploy()methodintheInstanceDeployer.Thismethoddeploysthecomputationaltasksonthesetofresourcesthatwerediscoveredduringtheinitializationphase.
9.5.2.3 Tracking/Steering a Deployed Application
The InstanceDeployer implements the JobLifecycleObserver interface, whichallowsonetotrackthestatusofmultiple jobs,andthecomputationaltasksthatcomprisethem.ClassesthatextendtheInstanceDeployerhavetheoptiontoover-ride methods specific to the JobLifecycleObserver interface. Specifically, for agiven Job,Granulesmaintains its registered JobLifecycleObserver and invokesmethodsonthisobserverwheneverthereisanupdatetothedeploymentorexecu-tionstatusofthecomputationaltasksthatcompriseit.
Associatedwitheach Job,GranulesmaintainsaProgressTracker thatmain-tainsinformationabouttheexecutionstateofeachofthecomputationaltasksthatcomprise the application. TheLifecycleMetrics associated with every computa-tionaltaskincludesinformationabout
1.Thearrivaltimeforthecomputationaltask 2.Thequeuingoverheadforthecomputationaltask 3.ThetotalCPU-boundtimeforthecomputationaltaskacrossmultipleitera-
tions(ifthereareany) 4.Theprocessingtimeforthecomputationaltask 5.The current status of the computational task {Awaiting Data,Queued for
Execution,Executing,Terminated,Successful,FAILED}
ThestatusofaJobisthecumulativestatusofthecomputationaltasksthatcom-priseit.
TheInstanceDeployeralsoincorporatesmethodsfortracking/steeringacom-putation.TherearemethodstorefreshthestatusofaspecificcomputationaltaskortheentireJob.Thesemethodsresultinupdatestothelife-cyclemetricsoftherelevant computational tasks. Additionally, Granules also allows computationaltaskstobeabortedwhentheyareinexecution.ThesystemallowseitheraspecificcomputationaltasktobesuspendedortheentireJob.
9.6 Related WorkTheoriginalMap-Reducepaper[1]byGhemawatandDeandescribedhowtheirprogrammingabstractionwasbeingusedintheGooglesearchengineandotherdata-intensiveapplications.Thisworkwasitselfinspiredbymapandreduceprimi-tivespresentinLispandotherfunctionalprogramminglanguages.GoogleMap-ReduceiswritteninC++withextensionsforJavaandPython.Sawzall[7]isan
K10175_C009.indd 215 3/2/2010 12:52:33 PM
216 ◾ Cloud Computing and Software Services: Theory and Practice
interpreted,proceduralprogramminglanguageusedbyGoogletodevelopMap-Reduceapplications.
Hadoop[8]wasoriginallydevelopedatYahoo,andisnowanApacheproject.ItisbyfarthemostwidelyusedimplementationoftheMap-Reduceframework.InadditiontothevastnumberofapplicationsatYahoo,itisalsopartoftheGoogle/IBM initiative to support university courses in distributed computing. Hadoopis also hosted as a framework over Amazon’s EC2 [9] cloud. Unlike Granules,Hadoopsupportsonlyexactly-oncesemantics,meaningthatthereisdirectsupportwithintheframeworkformapandreducefunctionstomaintainstate.
HadoopusestheHadoopDistributedFileSystem(HDFS)filesforcommuni-catingintermediateresultsbetweenthemapandreducefunctions,whileGranulesusesstreamingforthesedisseminations,thusallowingaccesstopartialresults.
HDFS allows for replicated, robust access to files. During the data-stagingphase,Hadoopallowsthecreationofreplicasonthe localfilesystem;computa-tionsarethenspawnedtoexploitdatalocality.Hadoopsupportsautomatedrecov-eryfromfailures.Currently,Granulesdoesnotincorporatesupportforautomatedrecoveryfromfailures;thiswillbethefocusofourfutureworkinthisarea.Here,weplantoharnessthereliablestreamingcapabilitiesavailableinNaradaBrokering.
The most dominant model for developing parallel applications in the HPCcommunityistheSPMD[2]model(firstproposedbyFedericaDarema)intandemwiththeMPI[10]library.TheSPMDmodelisapowerfulone,andMap-ReducecaninfactbethoughtofasaninstanceoftheSPMDmodel.TheuseofMPIhas,however,notbeenaswidespreadoutsidethescientificcommunity.
MicrosoftResearch’sDryad[11]isasystemdesignedasaprogrammingmodelfor developing scalable parallel and distributed applications. Dryad is based onDAGs.Inthismodel,sequentialprogramsareconnectedusingone-waychannels.It is intended tobea super-setof thecoreMap-Reduce framework.Dryadpro-videsjobmanagementandautonomiccapabilities,andmakesuseoftheMicrosoftSharedDirectoryService.However,sinceDryadisdevelopedbasedonDAGs,itisnotpossible todevelopsystemsthathavecycles inthem.Forexample, inourbenchmarks,wewerenotabletoimplementthek-meansmachinelearningalgo-rithm[12]usingthebasicDryadframework.
Phoenix[13]isanimplementationofMap-Reduceformulti-coreandmultipro-cessorsystems.ArelatedeffortisQtConcurrent[14],whichprovidesasimplifiedimplementationoftheMap-ReduceframeworkinC++.QtConcurrentautomati-cally optimizes thread utilizations on multi-core machines depending on coreavailability.Disco[15],fromNokia,isanopen-sourceMap-Reduceruntimedevel-opedusingtheErlangfunctionalprogramminglanguage.SimilartotheHadooparchitecture,DiscostorestheintermediateresultsinlocalfilesandaccessesthemusingHTTPconnectionsfromtheappropriatereducetasks.
Holumbus [16] includes an implementation of the Map-Reduce framework,developed in the Haskell functional programming language at the FH WedelUniversityofAppliedSciences,Germany.
AQ11
K10175_C009.indd 216 3/2/2010 12:52:33 PM
Granules ◾ 217
Skynet[17]isanopen-sourceRuby-basedimplementationoftheMap-Reduceframework. Skynet utilizes a peer-recovery system for tracking the constituenttasks.Peerstrackeachotherand,oncefailureisdetected,canspawnareplicaofthefailedpeer.
We had originally developed a prototype implementation of Map-Reduce,CGL-MapReduce [18], which implemented Map-Reduce using streaming (onceagain, using NaradaBrokering) with the ability to “keep alive” map instances.Granulesrepresentsanoverhaul,andincorporatesseveralnewcapabilities,suchasbuilt-insupportforsophisticatedlife-cyclemanagement(periodicity,datadriven,andterminationconditions),powerfulcreationandduplicatedetectionofresults,anddiagnosticsinadditiontotheabilitytocreatecomplexcomputationalpipelineswithfeedbackloopsinmultiplestages.ThecodebasefortheGranules(availablefordownload)runtimehasalsobeendevelopedfromscratch.
9.7 BenchmarksIn our benchmarks, we profile several aspects of the Granules’ performance.Wearespecifically interested indeterminingsystemperformancefordifferentlifecyclesassociatedwiththecomputationaltasks.Thedifferentlifecycleswebenchmarkincludeexactly-once,iterative,periodic,anddata-drivenprimitives.Where possible, we contrast the performance of Granules with comparablesystems, such as Hadoop, Dryad, and MPI. It is expected that these bench-markswouldbeindicativeoftheperformancethatcanbeexpectedindifferentdeployments.
All machines involved in these benchmarks have four dual-core CPUs, a2.4GHzclock,andan8GBRAM.Thesemachineswerehostedona100MbpsLAN.TheOperatingSystemonthesemachinesisRedHatEnterpriseLinuxver-sion4.AllJavaprocessesexecutedwithinversion1.6ofSun’sJVM.Weusedver-sion3.4.6ofthegcccomplierforC++,andforMPIweusedversion7.1.4oftheLocalAreaMulticomputer(LAM)MPI[19].
9.7.1 Streaming SubstrateSince we use the NaradaBrokering streaming substrate for all communicationsbetweenentities,wepresentasimplebenchmarktogivethereaderanideaofthecostsinvolvedinstreaming.Ourresultsoutlinethecommunicationlatenciesinasimplifiedsetting involvingoneproducer,oneconsumer,andonebroker.Thecommunication latencies are reported for stream fragments with different pay-loadsizes.AdditionalNaradaBrokeringbenchmarksindistributedsettingscanbefoundin[4,5].
Twoclustermachineswereinvolvedinthisbenchmark.Theproducerandcon-sumerwerehostedonthesamemachinetoobviatetheneedtoaccountforclock
AQ12AQ13
AQ14
K10175_C009.indd 217 3/2/2010 12:52:33 PM
218 ◾ Cloud Computing and Software Services: Theory and Practice
driftswhilemeasuringlatenciesforstreamsissuedbytheproducer,androutedbythebroker(hostedonthesecondmachine)totheconsumer.
Thereporteddelay, intheresultsdepictedinFigure9.8,istheaverageof50samplesforagivenpayloadsize,thestandarddeviationforthesesamplesalsobeingreported.TheY-axisforthestandarddeviationistheaxisontherightside(blue)ofthegraph.Streaminglatenciesvaryfrom750μs/hopfor100bytesto1.5ms/hopforastreamfragmentof10KBinclustersettings.
9.7.2 Information Retrieval: Exactly-OnceIn this section, we present results from a simple information retrieval example.Givenasetof textfiles, theobjective is tohistogramthecountsassociatedwithvariouswordsinthesefiles.TheperformanceofGranulesiscontrastedwiththatofHadoopandDryad.TheDryadversiontowhichwehaveaccessusesC#,LINQ,andfile-basedcommunicationsusingtheMicrosoftSharedDirectoryService.TheOSinvolvedintheDryadbenchmarksisWindowsXP.
Forthisbenchmark,wevarythecumulativesizeofthedatasetsthatneedtobeprocessed.Thetotalamountofdatathatisprocessedisvariedfrom20GBto100GB.Therewereatotalof128mapinstancesthatweredeployedonthefivemachinesinvolvedinthebenchmark.
The results depicted inFigure9.9demonstrate thebenefits ofusing stream-ingasopposedtofile-basedcommunications.Asthesizeofthedatasetsincreases,thereisaconcomitantincreaseinthenumberandsizeoftheintermediateresults(filebased).ThiscontributestotheslowerperformanceofHadoopandDryad.WeexpecttheperformanceofDryad’ssocket-basedversiontobefasterthantheirfile-basedversion.
0
1
2
3
4
5
100 1,000 10,0000
1
2
3
4
5M
ean
tran
sit d
elay
(ms)
Stan
dard
dev
iatio
n (m
s)
Content payload size (bytes)
Round-trip delays for different payload sizes (100 B–10 kB)
DelayStandard deviation
Figure 9.8 Streaming overheads in cluster settings.
K10175_C009.indd 218 3/2/2010 12:52:34 PM
Granules ◾ 219
9.7.3 K-Means: IterativeMachinelearningprovidesafertilegroundforiterativealgorithms.Inourbench-nmarks,we considered a simple algorithm in the areaofunsupervisedmachinelearning:k-means.Givenasetofndatapoints,theobjectiveistoorganizethesepointsintokclusters.
Thealgorithmstartsoffbyselectingkcentroids,andthenassociatesdifferentdatapointswithinthedatasettooneoftheclustersbasedontheirproximitytothecentroids.Foreachoftheclusters,newcentroidsarethencomputed.ThealgorithmissaidtoconvergewhenthecumulativeEuclideandistancebetweenthecentroidsinsuccessiveiterationsislessthanapredefinedthreshold.
Ink-means,thenumberofiterationsdependsontheinitialchoiceofthecen-troids,thenumberofdatapoints,andthespecifiederrorrate(signifyingthatthecentroidmovementsareacceptable).Theinitialsetofdatapointsisloadedateachofthemapfunctions.Eachmapisresponsibleforprocessingaportionoftheentiredataset.Whatchangesfromiterationtoiterationarethecentroids.Theoutputofeachmapfunctionisasetofcentroids.
The benchmarks, which were run on five machines, also contrast the per-formance of Granules with MPI using a C++ implementation of the k-meansalgorithm.
ThegraphsdepictedinFigure9.10havebeenplottedonalog-loggraphsothatthetrendscanbevisualizeda littlebetter.Wevariedthenumberofdatapoints in the dataset from 105 to 4×107. The results indicate that Hadoop’sperformanceisordersofmagnitudeslowerthanGranulesandMPI.InHadoop,
0
500
1000
1500
2000
2500
3000
20 30 40 50 60 70 80 90 100
Ove
rhea
d fo
r hist
ogra
mm
ing
wor
ds (s
)
Total size of dataset (GB)
HadoopDryad
Granules
Figure 9.9 Processing time for histogramming words.
K10175_C009.indd 219 3/2/2010 12:52:35 PM
220 ◾ Cloud Computing and Software Services: Theory and Practice
these centroids are transferred using files, while Granules uses streaming.Furthermore, since Hadoop does not support iterative semantics, map func-tionsneedtobeinitializedandthedatasetsneedtobereloadedusingHDFS.Though these file-system reads are being performed locally (thanks to HDFSanddata collocation), these costs can still beprohibitive, as evidenced inourbenchmarks.Additionally,asthesizeofthedatasetincreases,theperformancesoftheMPI/C++implementationofk-meansandtheGranules/Javaimplementa-tionofk-meansstarttoconverge.
9.7.4 Periodic SchedulingInthissection,webenchmarktheabilityofGranulestoperiodicallyscheduletasksforexecution.Forthisparticularbenchmark,weinitialized10,000mapfunctionsthatneededtobescheduledforexecutionevery4s.
TheobjectiveofthisbenchmarkistoshowthatasingleGranulesinstancecanindeedenforceperiodicityforareasonablenumberofmapinstances.
Figure9.11depictstheresultsofperiodicexecutionsof10,000mapsfor17iter-ations.Thegraphdepictsthespacinginthetimesatwhichthesemapsaresched-uledforexecution.TheX-axisrepresentsaspecificmapinstance(assignedIDsfrom1to10,000),andtheY-axisrepresentsthespacingbetweenthetimesatwhichagiveninstancewasscheduled.Eachmapinstancereports17values.
Thefirsttimeacomputationaltaskisscheduledforexecution,abasetime,tb,isrecorded.Subsequentiterationsreportthedifferencebetweenthebasetime,tb,andthecurrent time, tc. Inalmostallcases, thespacingbetweenthesuccessiveexecutions foranygiven instancewasbetween3.9–4.1s. Insomecases, there isasmallnotch;thisreflectscaseswherethefirstexecutionwasdelayedbyasmall
Ove
rhea
d (s
)
0.10.1
10000
1000
100
10
1
1 10 100Number of 2D data points (millions)
HadoopGranules
MPI
Figure 9.10 Performance of the k-means algorithm.
K10175_C009.indd 220 3/2/2010 12:52:36 PM
Granules ◾ 221
amount, the (constant) impactofwhich is reflected in subsequent iterations forthatmapinstance.
9.7.5 Data DrivenInthissection,wedescribetheperformanceofmatrixmultiplicationusingGranules.In this case, theobject is tomeasure theproductof twodense16,000×16,000matrices,thatis,eachmatrixhas256millionelementswithpredominantlynon-zerovalues.
Thematrixmultiplicationexampledemonstrateshowcomputationaltaskscanbe“stayalive,”andbescheduledforexecutionwhendataisavailable.Themapsarescheduledforexecutionasandwhenthedataisavailableforthecomputationstoproceed.
Forthisbenchmark,wevarythenumberofmachinesinvolvedintheexperi-ment from 1 to 8. There are a total of 16,000 map instances. At a given time,eachofthesemapsprocessesportionsoftherowsandcolumnsthatcomprisethematrix.EachGranulesinstancecopeswithafragmentofmorethan2000concur-rentstreams.Intotal,everyGranulesinstancecopeswith32,000distinctstreams.
The results for theprocessing times (plottedon a log-log scale) canbe seeninFigure9.12. Ingeneral, as thenumberof availablemachines increases, thereis a proportional improvement in the processing time. Our plots of the speed-up(Figure9.13)inprocessingtimeswiththeavailabilityofadditionalmachinesreflectthis.
Ingeneral,thesegraphsdemonstratethatGranulescanbringsubstantialben-efitstodata-drivenapplicationsbyamortizingthecomputationalloadonasetofmachines.Domainscientistsdonotneedtowriteasinglelineofnetworkingcode;Granulesmanagesthisinatransparentfashionfortheapplications.
0
10
20
30
40
50
60
70
0 2,000 4,000 6,000 8,000 10,000
Tim
e tas
k sc
hedu
led
at (s
)
Computational tasks
Figure 9.11 Periodic scheduling of 10,000 computational tasks.
K10175_C009.indd 221 3/2/2010 12:52:37 PM
222 ◾ Cloud Computing and Software Services: Theory and Practice
9.7.6 Assembling mRNA SequencesThissectiondescribestheperformanceofGranulesinorchestratingtheexecutionofapplicationsdevelopedinlanguagesotherthanJava.TheapplicationweconsideristheCAP3[20]messengerRibonucleicacid(mRNA)sequenceassemblyapplica-tion(C++)developedatMichiganTech.
AsExpressedSequenceTag (EST)corresponds tomRNAs transcribed fromgenes residingonchromosomes, individualEST sequences represent a fragmentofmRNA.CAP3allowsus toperformESTassembly to reconstruct full-lengthmRNAsequencesforeachexpressedgene.
OurobjectiveaspartofthisbenchmarkwasalsotoseehowGranulescanbeusedtomaximizecoreutilizationsonamachine.CAP3takesasinputasetoffiles.Inourbenchmark,weneedtoprocess256filesduringtheassembly.
AQ15
1024
2048
4096
8192
16384
1 2 4 8
Proc
essin
g tim
e (s)
Number of machines
Processing time
Figure 9.12 Processing time for matrix multiplication on different machines.
1
2
4
8
1 2 4 8
Spee
d-up
Number of machines
Speed-up
Figure 9.13 Speed-up for matrix multiplication.
K10175_C009.indd 222 3/2/2010 12:52:38 PM
Granules ◾ 223
Onagivenmachine,wefine-tunedtheconcurrencybysettingthenumberofworkerthreadswithinthethread-pooltodifferentvalues.Byrestrictingthenum-berofthreads,wealsorestrictedtheamountofconcurrencyandtheunderlyingcoreutilizations.Westartedoffbysettingtheworker-poolsizeto1,2,4,and8on1machine,andthenused8workerthreadson2,4,and8machines.Thisallowedustoreportresultsfor1,2,4,8,16,32,and64cores.
Theresultsofourbenchmarkintermsofprocessingcostsandthespeed-upsachievedaredepictedinFigures9.14and9.15,respectively.Ingeneral,asthenum-berofavailablecoresincreases,thereisacorrespondingimprovementinexecutiontimes.
8
16
32
64
128
256
512
1024
1 2 4 8 16 32 64
Proc
essin
g tim
e (s
)
Number of cores
Processing time
Figure 9.14 Processing time for EST assembly on different cores using Granules and CAP3.
1
2
4
8
16
32
64
1 2 4 8 16 32 64
Spee
d-up
Number of cores
Speed-up
Figure 9.15 Speed-up for EST assembly using Granules and CAP3.
K10175_C009.indd 223 3/2/2010 12:52:39 PM
224 ◾ Cloud Computing and Software Services: Theory and Practice
Theresultsdemonstratethat,whenconfiguredcorrectly,Granulescanmaxi-mizecoreutilizationsonagivenmachine.Thegraphsplottedona log-logscaleindicate that for every doubling of the available cores, the processing time forassemblingthemRNAsequencesreducesbyhalf(approximately).Currently,theGranulesruntimereadythethread-poolsizinginformationfromaconfigurationfile;wewill be investigatingmechanisms thatwill allowus todynamically sizethesethread-pools.
9.8 ConclusionsInthischapter,wedescribedtheGranulesruntime.Supportforrichlife-cyclesup-portwithinGranulesallowscomputationstoretainstate,whichinturnisparticu-larlyapplicableforseveralclassesforscientificapplications.
Granules allows complex computational graphs to be created. As discussed,thesegraphscanencapsulatebothcontrolflowanddataflow.Granulesenforcesthesemanticsofcomplexdistributedcomputationalgraphsthathaveoneormorefeedback loops.Thedomainscientistdoesnothave tocopewithIO, threading,synchronization,ornetworking librarieswhiledevelopingapplications that spanmultiplestages,withmultipledistributedinstancescomprisingeachstage.Thesecomputationalpipelines canbedependenton iterative,periodic,data-driven,orterminationconditions.
DemonstrableperformancebenefitshavebeenaccruedbyGranulesasaresultofusingstreamingfordisseminatingintermediateresults.
Granules’ rich life-cycle support, and its performance when contrasted withcomparable systems,underscores the feasibilityofusingGranules in several set-tings.Aspartofourfuturework,wewillbeinvestigatingsupportforautonomicerrordetectionandrecoverywithinGranules.
References 1. J.DeanandS.Ghemawat,Mapreduce:Simplifieddataprocessingonlargeclusters,
Communications of the ACM,51,107–113,January2008. 2. F.Darema,SPMDmodel:Past,presentandfuture,Recent Advances in Parallel Virtual
Machine and Message Passing Interface: Eighth European PVM/MPI Users’ Group Meeting,Santorini/Thera,Greece,2001.
3. S.Pallickara,J.Ekanayake,andG.Fox,Anoverviewofthegranulesruntimeforcloudcomputing(ShortPaper),Proceedings of the IEEE International Conference on e-Science,Indianapolis,IN,2008.
4. S.Pallickara andG.Fox,Naradabrokering:Amiddleware framework and architec-ture for enabling durable peer-to-peer grids, Proceedings of the ACM/IFIP/USENIX International Middleware Conference Middleware-2003,RiodeJaneiro,Brazil,2003,pp.41–61.
AQ16
AQ17
K10175_C009.indd 224 3/2/2010 12:52:39 PM
Granules ◾ 225
5. S.Pallickaraetal.,Aframeworkforsecureend-to-enddeliveryofmessagesinpublish/subscribe systems,Proceedings of the Seventh IEEE/ACM International Conference on Grid Computing(GRID 2006),Barcelona,Spain,2006.
6. S.Pallickara,H.Bulut,andG.Fox,Fault-tolerantreliabledeliveryofmessagesindis-tributedpublish/subscribesystems,Fourth IEEE International Conference on Autonomic Computing,Jacksonville,FL,June2007,p.19.
7. R.Pike,S.Dorward,R.Griesemer,andS.Quinlan,Interpretingthedata:Parallelanal-ysiswithSawzall,Scientific Programming Journal,SpecialIssueonGridsandWorldwideComputingProgrammingModelsandInfrastructure,13(4),227–298,2005.
8. ApacheHadoop,http://hadoop.apache.org/core/ 9. S.Garfinkel,Anevaluationofamazon’sgridcomputingservices:EC2,S3andSQS,
TechnicalReportTR-08-07,HarvardUniversity,Cambridge,MA,August2007. 10. Message Passing Interface Forum, MPI: A message passing interface, Proceedings of
Supercomputing’93,Portland,OR, IEEEComputerSocietyPress,Washington,DC,November1993,pp.878–883.
11. M.Isard,M.Budiu,Y.Yu,A.Birrell,andD.Fetterly,Dryad:Distributeddata-parallelprogramsfromsequentialbuildingblocks,European Conference on Computer Systems,Lisbon,Portugal,March2007.
12. J.B.MacQueen,Somemethodsforclassificationandanalysisofmultivariateobserva-tions,Proceedings of Fifth Berkeley Symposium on Mathematical Statistics and Probability,Vol.1,Berkeley,CA,UniversityofCaliforniaPress,Berkeley,CA,1967,pp.281–297.
13. C.Ranger,R.Raghuraman,A.Penmetsa,G.R.Bradski,andC.Kozyrakis,Evaluatingmapreduceformulti-coreandmultiprocessorsystems,Proceedings of the International Symposium on High-Performance Computer Architecture(HPCA),Phoenix,AZ,2007,pp.13–24.
14. QtConcurrent,SimplifiedmapreduceinC++withsupportformulticores,April2009,http://labs.trolltech.com/page/Projects/Threads/QtConcurrent
15. Discoproject,http://discoproject.org/ 16. S.Schlatt,T.Hübel,S.Schmidt,andU.Schmidt,TheHolumbusdistributedcomput-
ingframeworkandmapreduceinHaskell,2009,http://holumbus.fh-wedel.de/trac 17. A.Pisoni,Skynet:Arubymapreduceframework,April2009,http://skynet.rubyforge.org/ 18. J. Ekanayake, S. Pallickara, and G. Fox, Map-reduce for scientific applications,
Proceedings of the IEEE International Conference on e-Science,Indianapolis,IN,2008. 19. J.M.SquyresandA.Lumsdaine,AcomponentarchitectureforLAM/MPI,Proceedings
of Euro PVM/MPI,Venice,Itlay,October2003. 20. X. Huang and A. Madan, CAP3: A DNA sequence assembly program, Genome
Research,9,868–877,1999. 21. S.Pallickara,J.Ekanayake,andG.Fox,Granules:Alightweight,streamingruntimefor
cloudcomputingwithsupportformap-reduce,Proceedings of the IEEE International Conference on Cluster Computing(CLUSTER 2009),NewOrleans,LA,2009.
AQ18
K10175_C009.indd 225 3/2/2010 12:52:39 PM
AUTHORQUERIES[AQ1] Pleasecheckifthefixedrunningheadisok.[AQ2] Inthetext,“italics”hasbeenusedforemphasis.Pleasecheckwhetherthis
canbechangedtobold.[AQ3] Pleaseexpand“SPMD,”ifappropriate.[AQ4] Pleasecheckcrossreferencetosectionhereandinotherinstances.[AQ5] SinceanintroductiontoSection9.5wasmissing,onehasbeenaddedhere.
Pleasecheckifthisisfine.[AQ6] Pleaseexpand“GIS,”ifappropriate.[AQ7] Pleaseexpand“JDBC,”ifappropriate.[AQ8] Pleaseexpand“FIFO,”ifappropriate.[AQ9] Pleasecheckifthechangemadetothesentencestarting“Granulessup-
ports…”isokay.[AQ10] Pleaseexpand“SSL,”ifappropriate.[AQ11] Pleaseexpand“HPC,”ifappropriate.[AQ12] Pleasecheckiftheeditofthesentencestarting“Wehadoriginally…”is
correct.[AQ13] Pleasecheckwhether“CGL-MapReduce”shouldbechangedto“CGL-
Map-Reduce”asperotheroccurrencesofMap-Reduceinthischapter.[AQ14] Pleaseexpand“JVM,”ifappropriate.[AQ15] Please check if the edit of the sentence: “As expressed… fragment of
mRNA”iscorrect.[AQ16] Pleasecheckif“ready”shouldbechangedto“reads”inthesentencestart-
ing“Currently,theGranules…”[AQ17] Pleasecheckwhetherthephrase“Supportforrichlife-cyclesupport”can
bechangedto“Richlife-cyclesupport”inthesentencestarting“Supportfor…”
[AQ18] Pleasechecktheinsertedlocationofthepublisherforcorrectness.
K10175_C009.indd 226 3/2/2010 12:52:39 PM