domino performance - best practicesgcc.uni-paderborn.de/www/wi/wi2/wi2_lit.nsf... · domino memory...

Domino Performance -Best Practices

Copyright (c) 2004 Kim Greene Consulting, Inc.. All rights reserved worldwide.

Kim [email protected]

2Copyright © 2004 Kim Greene Consulting, Inc. All Rights Reserved Worldwide

Introduction - Kim Greene

President of Kim Greene ConsultingPresident of Kim Greene ConsultingServices offered include:Services offered include:ƒƒSystem and application tuningSystem and application tuningƒƒEnterprise integrationEnterprise integrationƒƒDomino developmentDomino developmentƒƒCustomized education and trainingCustomized education and trainingƒƒTechnical writingTechnical writing

Member of Penumbra GroupMember of Penumbra Groupƒƒwww.penumbra.orgwww.penumbra.org


What We'll Cover...

Domino memory management Domino memory management ƒƒNSF buffer poolNSF buffer poolƒƒDBcacheDBcacheƒƒDpoolDpool

Mail file performanceMail file performanceƒƒSize of mail filesSize of mail filesƒƒImpact from number of documentsImpact from number of documentsƒƒFull text indexes and searchingFull text indexes and searchingƒƒQuickRulesQuickRules


What We'll Cover...

Additional performance best practicesAdditional performance best practicesƒƒLPARLPARƒƒDomino release levelsDomino release levelsƒƒTransaction loggingTransaction logging


Domino Memory Management



How Domino divides main storageHow Domino divides main storageƒƒDomino has it’s own memory pools/buffersDomino has it’s own memory pools/buffersƒƒMemory subpools are dynamically adjusted by Memory subpools are dynamically adjusted by DominoDomino

NSF Buffer

DB Cache

Apps Users



NSF Buffer Pool NSF Buffer Pool –– what it doeswhat it does

NSF Buffer PoolNIF Disk



NSF Buffer Pool NSF Buffer Pool –– how size is sethow size is setƒDomino R5

ƒSet to 300 MB by default on iSeriesƒDomino 6

ƒSet to 748 MB by default on iSeriesƒƒRecommend setting based on actual server Recommend setting based on actual server needsneedsƒƒUseUse Show Stat DatabaseShow Stat Database command to view command to view statisticsstatistics



Domino database statisticsDomino database statisticsDatabase.Database.BufferPool.Maximum.MegabytesDatabase.Database.BufferPool.Peak.MegabytesDatabase.Database.BufferPool.PercentReadsInBuffer

Want >= 95%

Database.DbCache.CurrentEntriesNumber of entries in the database cache

Database.DbCache.HighWaterMarkPeak number of entries in db cache since Domino server was started

Database.DbCache.MaxEntriesNumber of databases that can be cached

Database.DbCache.OvercrowdingRejections



Customer example Customer example –– looking goodlooking good

Database.Database.BufferPool.Maximum.Megabytes = 710Database.Database.BufferPool.Peak.Megabytes = 708Database.Database.BufferPool.PercentReadsInBuffer = 98.12Database.DbCache.CurrentEntries = 71Database.DbCache.HighWaterMark = 256Database.DbCache.MaxEntries = 300Database.DbCache.OvercrowdingRejections = 0



Customer example Customer example –– looking very bad looking very bad Database.Database.BufferPool.Maximum.Megabytes = 170Database.Database.BufferPool.Peak.Megabytes = 168Database.Database.BufferPool.PercentReadsInBuffer = 95.44Database.DbCache.CurrentEntries = 510Database.DbCache.MaxEntries = 510Database.DbCache.OvercrowdingRejections = 8385

Huge problem!

Options:Increase NSF buffer pool sizeSet NSF_DbCache_MaxEntries



NSF_Buffer_Pool_Size_MBNSF_Buffer_Pool_Size_MBƒƒ What to monitor when fine tuningWhat to monitor when fine tuning

1. Database.BufferPool.PercentReadsInBuffer >= 95%2. Non-database page faulting = < 100 faults/processor3. Database cache size

° Database.DbCache.MaxEntries = 3 X BufferPool4. View sizes

° Faulting can occur with NSF buffer pool set too small



Two key notes.ini parameters affect Two key notes.ini parameters affect Domino memory managementDomino memory management

ƒƒNSF_Buffer_Pool_Size_MBNSF_Buffer_Pool_Size_MBƒƒPercentAvailSysResources PercentAvailSysResources

Other Domino memory buffers are Other Domino memory buffers are allocated based on these parametersallocated based on these parameters



PercentAvailSysResourcesPercentAvailSysResourcesƒƒIntroduced in R5.0.4Introduced in R5.0.4ƒƒControls % of memory resources Controls % of memory resources available to Domino serveravailable to Domino server

oValid values = 2% to 100%o Memory available depends on where Domino server is executing

ƒƒAllows dynamic adjustment of buffer Allows dynamic adjustment of buffer sizessizes



NSF_Buffer_Pool_Size_MBNSF_Buffer_Pool_Size_MBƒƒStatic setting of NSF buffer poolStatic setting of NSF buffer poolƒƒUse in combination with Use in combination with PercentAvailSysResourcesPercentAvailSysResources°PercentAvailSysResources won’t override NSF_Buffer_Pool_Size_MB setting



Notes_SHARED_DPOOLSIZENotes_SHARED_DPOOLSIZEƒƒDPOOL is memory pool used by DominoDPOOL is memory pool used by DominoƒƒControls minimum memory size allocated by Domino Controls minimum memory size allocated by Domino from the DPOOLfrom the DPOOL

ƒƒUse Domino console command Use Domino console command SHOW MEMORY SHOW MEMORY DUMPDUMP

°Dumps information to a stream file°Look for the following:

Number of DPOOLs Percent used in each DPOOLNumber of free space searchesNumber of search failures



Two ways to set the Dpool allocation sizeTwo ways to set the Dpool allocation sizeƒƒSet variable in notes.ini fileSet variable in notes.ini file

°Notes_SHARED_DPOOLSIZE=1048576Sets for specific Domino server

ƒƒAdd environment variable (iSeries):Add environment variable (iSeries):°ADDENVVAR ENVVAR(Notes_SHARED_DPOOLSIZE) VALUE(1048576) LEVEL(*SYS)

Sets for ALL Domino servers


Mail Performance


Watch Size of Mail Files

Mail file size can have dramatic impact Mail file size can have dramatic impact on performanceon performance100 MB mail file as baseline100 MB mail file as baseline

ƒƒAdditional 10% CPU for each 100 MBAdditional 10% CPU for each 100 MBƒƒHit knee of curve at 500 MBHit knee of curve at 500 MB

Size of Mail File CPU consumption

100 MB x%200 MB x + 10%300 MB x + 20%400 MB x + 30%500 MB exponential, hit knee of curve


Impact of Number of Documents

Number of documents has greater impact Number of documents has greater impact as file size growsas file size grows

0

2

4

6

8

10

12

14

16

100 MB 700 MB

1,024 docs4,096 docs7,196 docs28,672 docs


File Those Documents!

Inbox is most expensive container in a Inbox is most expensive container in a mail filemail fileƒƒUpdated each time user opens mailUpdated each time user opens mailƒƒUpdated each time user clicks Refresh Updated each time user clicks Refresh buttonbutton

Reducing number of documents in Inbox Reducing number of documents in Inbox reduces:reduces:ƒƒCPU utilizationCPU utilizationƒƒMain storage required to update the viewMain storage required to update the view


File Those Documents!

Reducing number of documents stored Reducing number of documents stored in Inboxin Inbox

°Mail file open performance improved 50%°Steady state performance improved 12%

0

5

10

15

20

25

30

25% in Inbox 100% in Inbox

OpenSteady State


Mail File Best Practices

For optimal performanceFor optimal performanceƒƒReduce overall document count in Reduce overall document count in the filethe file

••Up to 2X CPU reductionUp to 2X CPU reductionƒƒKeep number of documents stored in Keep number of documents stored in Inbox to a minimumInbox to a minimum


Full-Text Indexing and Searches

Full text indexing is cheapFull text indexing is cheapƒƒCPU usage increase < .5%CPU usage increase < .5%

Searches are expensiveSearches are expensiveƒƒ20% extra CPU20% extra CPU

Moral of the storyMoral of the storyƒƒCreate permanent indexes if using FTI Create permanent indexes if using FTI functionfunction


Mail Statistics to Watch

Mail.WaitingMail.WaitingƒƒIndicator of ROUTER performanceIndicator of ROUTER performanceƒƒNumber of mail items waiting to be Number of mail items waiting to be delivered in all MAIL.BOX filesdelivered in all MAIL.BOX files°May indicate need additional files

Mail.TotalPendingMail.TotalPendingƒƒNumber of mail messages resident in all Number of mail messages resident in all MAIL.BOX filesMAIL.BOX files°Includes all dead, held, and pending mail


Limit Use of QuickRules

QuickRulesQuickRulesƒƒIntroduced in Notes 6.5Introduced in Notes 6.5ƒƒRules applied to mail coming into InboxRules applied to mail coming into Inbox

°Often used to reduce Junk mail in InboxƒƒCan cause ROUTER CPU to spikeCan cause ROUTER CPU to spike

°LIMIT USE UNTIL PERFORMANCE PROBLEM IS FIXED!!!


Additional PerformanceBest Practices


Performance Impact of LPAR

Logical partitioning (LPAR)Logical partitioning (LPAR)ƒƒAbility to carve physical machine up into Ability to carve physical machine up into smaller systemssmaller systems

ƒƒMinimumMinimum LPAR CPU count should be LPAR CPU count should be oneone°Example: LPAR with .5 processor

Only get CPU for processing 1/2 of clock timeCan greatly impact performance, because waiting to get CPU 50% of time


Performance Impact of LPAR

Logical partitioning (LPAR)Logical partitioning (LPAR)ƒƒNo noticeable impact from LPAR No noticeable impact from LPAR performanceperformance°6-way has comparable performance to a 6-way LPAR configuration


Domino Release Levels

It pays to stay current with Domino It pays to stay current with Domino release levelsrelease levelsƒƒR5 R5 --> D6> D6

°Up to 30% improvementƒƒ6.0.16.0.1--> 6.0.2> 6.0.2

°° 20% improvement for Domino Web Access20% improvement for Domino Web Accessƒƒ6.0.2 6.0.2 --> 6.5> 6.5

°~6% better performance


Transaction Logging

Transaction loggingTransaction loggingƒƒUse when need enhanced failover Use when need enhanced failover capabilitiescapabilities

ƒƒDoes impact CPUDoes impact CPU°°Additional 15% CPU utilizationAdditional 15% CPU utilization

ƒƒAlso impacts end user response timeAlso impacts end user response timeƒƒCircular log helps limit performance Circular log helps limit performance impactimpact

°°Keeps size of log file cappedKeeps size of log file capped


Transaction Logging Impact

Impact of transaction loggingImpact of transaction logging

Number of users No logging Circular logging Archive logging

3,000 17 milliseconds 17 milliseconds 66 milliseconds



15,000 38 milliseconds 1.266 seconds 1.892 seconds


References


References

Performance ZonePerformance Zoneƒƒwww.lotus.com/performancewww.lotus.com/performance

Lotus Developer DomainLotus Developer Domainƒƒwww.lotus.com/lddwww.lotus.com/ldd

Ask Professor INIAsk Professor INIƒƒwww.lotus.com/ldd/today.nsf/profini?OpenViewwww.lotus.com/ldd/today.nsf/profini?OpenView

LDD TodayLDD Todaywww.lotus.com/ldd/today.nsfwww.lotus.com/ldd/today.nsf

Iris SandboxIris SandboxPerformance presentationsPerformance presentations


References

iSeries Domino PerformanceiSeries Domino Performanceƒƒwww.ibm.com/servers/enable/site/domino/perform.htmlwww.ibm.com/servers/enable/site/domino/perform.html

IBM Workload EstimatorIBM Workload Estimatorƒƒwwwwww--912.ibm.com/servlet/EstimatorServlet912.ibm.com/servlet/EstimatorServlet

Domino for iSeries Sizing and Performance Domino for iSeries Sizing and Performance Tuning on the IBM Tuning on the IBM eServereServer iSeries ServeriSeries Server

ƒƒwww.ibm.com/redbookswww.ibm.com/redbooks°°SG24SG24--51625162--0202


Domino Performance Redpapers

Sizing LargeSizing Large--Scale Domino Workloads on iSeriesScale Domino Workloads on iSeriesƒƒ publibpublib--b.boulder.ibm.com/Redbooks.nsf/RedpaperAbstracts/redp3802.html?Ob.boulder.ibm.com/Redbooks.nsf/RedpaperAbstracts/redp3802.html?Openpen

IBM Lotus Domino for iSeries Performance and IBM Lotus Domino for iSeries Performance and Tuning, REDP3843Tuning, REDP3843

ƒƒwww.redbooks.ibm.com/redpapers/pdfs/redp2843.pdfwww.redbooks.ibm.com/redpapers/pdfs/redp2843.pdf


Your Turn!

Questions?