![Page 1: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/1.jpg)
OpenLDAP
Key NotesHoward Chu, [email protected] Architect, Symas Corp
Chief Architect, OpenLDAP ProjectSambaXP 2007-04-24
![Page 2: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/2.jpg)
OpenLDAP Project• Open source code project• Founded 1998• Three core team members• A dozen or so contributors• Feature releases every 12-18 months• Maintenance releases roughly monthly
![Page 3: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/3.jpg)
OpenLDAP Releases• Release 1.x 1998/08/08 – 2001/09/11
– Basically the UMich 3.3 code– Supported LDAP version 2
• Release 2.0 2000/08/31 – 2002/09/22– Introduced LDAP version 3 support– Added security with SASL and SSL/TLS
• Release 2.1 2002/06/09 – 2004/04/15– Significantly faster than 2.0– Added Unicode support– Added back-bdb backend
![Page 4: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/4.jpg)
OpenLDAP Releases, cont'd• Release 2.2 2003/12/31 – 2005/11/21
– Further optimization– Added back-hdb– More extensibility using slapd overlays and/or
SLAPI plugins
• Release 2.3 2004/12/30 – now– Component-based matching– More overlays– Dynamic reconfiguration
![Page 5: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/5.jpg)
OpenLDAP Today• The fastest Directory Server• The most reliable• The most scalable• The most active Open Source DS project• The most aggressive, looking forward
![Page 6: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/6.jpg)
A Word About Symas• Founded 1999• Founders from Enterprise Software world
– platinum Technology (Locus Computing)– IBM
• Howard joined OpenLDAP in 1999– One of the Core Team members– Appointed Chief Architect January 2007
![Page 7: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/7.jpg)
Notable Features
• Introduced in 2.1:– Transactional Backend (back-bdb)
• Introduced in 2.2:– Hierarchical Backend (back-hdb)– Content-Sync Replication (syncrepl)– Overlays
• Introduced in 2.3:– Dynamic Configuration (back-config)
![Page 8: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/8.jpg)
back-bdb• Fully transactional backend with full ACID
semantics– Atomicity: changes are all-or-nothing– Consistency: no structural corruption– Isolation: no in-between views of data– Durability: once a write returns success, it cannot
be undone
• Extremely reliable
![Page 9: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/9.jpg)
back-hdb• Fully hierarchical backend
– Higher write throughput than other directory backends
– Supports subtree renames in O(1) time– Based on back-bdb code – offers same
transactional reliabilty
![Page 10: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/10.jpg)
syncrepl• Replaces the old slurpd-based replication
mechanism• Documented in RFC4533• Very flexible operation with minimal
administration overhead
![Page 11: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/11.jpg)
Slapd overlays• Modular plugin framework using slapd's
native API• Allows for rapid development and
deployment of enhancements and new features
![Page 12: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/12.jpg)
Overlay Examples
• Enterprise-oriented features– In-directory password policy– Referential integrity– Translucency– Attribute uniqueness– Value sorting– In-directory logging
![Page 13: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/13.jpg)
Dynamic Configuration• cn=config database
– Config engine is backward compatible with slapd.conf
– Allows runtime changes of almost all settings• ACLs• Schema• Databases• DB indexing• Dynamic modules
– Changes take effect immediately, no downtime required
![Page 14: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/14.jpg)
CN=config Future
• Zero administrative downtime– dynamically replace/re-exec binaries
• Fine-grained syncrepl for shared configuration components– Available in OpenLDAP 2.4
• config_entry API– allow backends/overlays to access their own
config entries and persist private state
• Your suggestions welcome...
![Page 15: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/15.jpg)
New Developments
• Syncrepl enhancements– Delta-syncrepl– Push-mode syncrepl– Mirrormode
• Upcoming work– lessons learned from deployment, ITS’s
![Page 16: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/16.jpg)
Syncrepl
• Delta-syncrepl– Addresses bandwidth concerns from plain
syncrepl– Relies on a persistent log of changes– Ordering of log entries is fully serialized; no out
of order updates– Automatic fallback to plain syncrepl if consumer
loses sync with log
![Page 17: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/17.jpg)
Syncrepl...
• Push-mode syncrepl– Just a syncrepl consumer sitting on back-ldap– Can add a customization overlay for mapping
the contextCSN to a suitable remote attribute, or to store the contextCSN locally
– Provides a simple, robust, dynamically configurable replacement for slurpd
![Page 18: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/18.jpg)
Syncrepl...• Mirrormode
– Allows a single active master and many standby masters– Preserves single master consistency while allowing
automatic promotion of alternate masters– Requires use of an external frontend to guarantee that
writes are only sent to a single master at a time– Addresses the high availability/SPOF concerns with
minimal fuss– Already in use at some Symas customer sites
![Page 19: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/19.jpg)
Syncrepl...
• Full N-Way Multimaster Support– requires synchronized clocks for all contexts– requires use of hostID field of CSN– requires per-consumer contextCSNs in addition
to (*not instead of*) provider contextCSN– Available in OpenLDAP 2.4
![Page 20: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/20.jpg)
Performance
• Fixed Lightweight Dispatcher– eliminated unnecessary locking in connection
manager• slapd-auth test against back-null yielded over 32000
binds per second on 100Mbps ethernet• over 128000 frames per second - ~90% of available
bandwidth – essentially saturated• No other LDAP server we tested delivers this speed
on identical hardware
![Page 21: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/21.jpg)
Performance...
• Fixes to pcache (proxy cache) overlay– Fixed O(n^2) query containment behaviors– Optimized case where a single entry is expected– Added negative caching support– Results:
• pcache used to be slower than a direct proxy lookup above about 500 queries
• pcache is now always faster than passing through
![Page 22: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/22.jpg)
Performance...
• libc malloc() still has a major impact– refactored Entry and Attribute management to
further reduce number of calls to malloc– using a thread-oriented allocator like hoard
provides further advantages
![Page 23: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/23.jpg)
malloc Performance
see openldap–devel August 30 2006...
StartSingle
SingleSingle
SingleFour
FourFour
Four
0
250000
500000
750000
1000000
1250000
1500000
1750000
2000000
2250000
00:00.00
00:17.28
00:34.56
00:51.84
01:09.12
01:26.40
01:43.68
02:00.96
02:18.24
02:35.52
02:52.80
Malloc performance
Glibc size
Glibc time
Hoard size
Hoard time
Umem size
Umem time
Tcmalloc size
Tcmalloc timePro
cess
siz
e, K
B
![Page 24: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/24.jpg)
malloc Performance
• Tested on 2.6 Linux kernel with glibc 2.3.3• Results will obviously vary by platform• glibc malloc does not handle tight memory
conditions gracefully• libhoard is marginally fastest• Google tcmalloc is most space-efficient• umem on non-Solaris appears unmaintained
![Page 25: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/25.jpg)
Performance...
250K1M
2.5M
0
2000
4000
6000
8000
10000
12000
140001500016000
15906.75
15374.795
13391.246
4525.744
4304.435
3548.711
4178.456
3993.396
3610.191
635.85
Authentication Rate
ApacheDS 1.0.1
OpenDS 0.1-34FedoraDS 1.0.4
OpenLDAP 2.3.34
DB Entries
Auth
s/s
ec
![Page 26: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/26.jpg)
Performance...
250K1M
2.5M
0
10000
20000
30000
40000
500005500060000650007000075000
71633
67249
27044
48923
102969790
92318666
82671680
Search, entries per second
ApacheDS 1.0.1
OpenDS 0.1-34
FedoraDS 1.0.4OpenLDAP 2.3.34
DB Entries
![Page 27: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/27.jpg)
Performance...
250K1M
2.5M
00:00.00
00:43.20
01:26.40
02:09.60
02:52.80
03:36.00
04:19.2004:40.80
02:28.78
00:27.08
01:55.39
05:02.38
00:05.11
01:37.12
04:15.35
00:03.4900:14.87
01:32.44
Search Times
(lower is better)
OpenLDAP 2.3.34
FedoraDS 1.0.4
OpenDS 0.1-34
ApacheDS 1.0.1
DB Entries
Tim
e
![Page 28: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/28.jpg)
Performance...
250K
1M
2.5M
00:00.00
07:12.00
14:24.00
21:36.00
28:48.00
36:00.0039:36.0043:12.0046:48.0050:24.0054:00.00
50:51.05
01:51.2008:46.13
20:23.51
01:04.10 04:59.64 11:33.93
00:45.0003:18.18 08:23.84
DB Load Times(lower is better)
OpenLDAP 2.3.34
FedoraDS 1.0.4
OpenDS 0.1-34
ApacheDS 1.0.1
DB Entries
Tim
e
![Page 29: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/29.jpg)
Performance Notes...• No, it's not rigged
– Verified optimal FDS config with Rich Megginson, FDS lead developer
– Verified optimal ODS config with Neil Wilson, ODS lead developer
![Page 30: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/30.jpg)
Performance Notes...• FDS published documentation and expert
advice are wrong– FDS defaults to 30 server threads, and docs advise
using more threads for heavier loads.– Optimal performance achieved with 8 threads,
10% faster.
![Page 31: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/31.jpg)
Performance Notes...• Sun docs and experts are wrong
– ODS has no viable entry cache: “Experience indicates that entry cache with multi-gigabyte databases is ineffective”
• OpenLDAP clearly proves that entry caching can be effective at any DB size. Sun advice contradicts all current wisdom in computer architecture. One need only look at CPUs with L1, L2, and L3 caches...
• Their experience only reflects that their entry cache is so ineffective.
![Page 32: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/32.jpg)
Performance Notes...• More from Sun
– Their docs and experts say the BerkeleyDB cache size is unimportant, since the filesystem cache will take care of things if the BDB cache is too small
• But relying on the FS cache requires an extra memory copy operation, as well as twice as much RAM.
• 30% performance gain by ignoring their advice.
![Page 33: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/33.jpg)
Performance...• Scaling to large deployments
– Demonstrated performance at over 150 million entries• November 2005: 16600 queries/second, 3400 updates/second• April 2006: 22000 queries/second, 4800 updates/second
– Over 1 terabyte of real data– Other popular directories’ claims of scaling are provably
false• Several other products were tested with the same data, all of
them failed to meet the test requirements• Only OpenLDAP passed
![Page 34: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/34.jpg)
Performance...• Scaling to multiple CPUs
– Single client, ldapsearch across entire 1.3GB database in 0.70 seconds
– Two clients concurrent, ldapsearch in 0.71 seconds
– 4 clients, 1.42 seconds– Practically Perfect Parallel Scaling– (Using AMD64 X2 dual-core processor)
![Page 35: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/35.jpg)
Performance...
• benchmark details available on www.symas.com
• we may want to consider investing effort in a C-based benchmarking framework– existing frameworks are not credible
• DirectoryMark in perl, fast enough to measure slow directories, not fast enough for OpenLDAP
• SLAMD in java, same story again
![Page 36: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/36.jpg)
The Lay of the Land
• OpenLDAP is the only directory software that matters– Increasing wins in telecoms, government, higher
education, Fortune 100• Is now running all of HP’s corporate IT, displacing
previous proprietary server
– Feature wise, performance wise, and on raw expertise, there is no credible competition
![Page 37: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/37.jpg)
The Big Picture• It's more than just standards, protocols, and
APIs• Interoperability starts with people -
communicating, contributing, cooperating– Clearly the Samba team already gets this – big
Thank You to all– Some companies are still learning...
• Apple OpenDirectory, IBM Tivoli, etc...
![Page 38: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/38.jpg)
The Road Ahead
• The unmatched code quality is not matched by documentation quality– Working on OpenLDAP Admin book, to be
published by Addison-Wesley in Summer 2007– The Admin Guide needs to be expanded
![Page 39: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/39.jpg)
The Road Ahead...
• Work on scale-out, vs scale-up– allow multi-terabyte DBs to be served without
requiring a single giant server• page-oriented, lock-free DB to allow multiple
backends to serve portions of a single shared DB• distributed indexing• cluster-friendly optimizations
![Page 40: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/40.jpg)
Final Thoughts• OpenLDAP is taking over the enterprise
– reliability, flexibility, scalability beyond all users’ or competitors’ comprehension
• Code quality is self-evident, but needs to be balanced with documentation quality
• The OpenLDAP community continues to thrive– When we work together everyone wins
![Page 41: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/41.jpg)
Authentication PerformanceA u t h e n t i c a t i o n R a t e s
0
1 0 0 0
2 0 0 0
3 0 0 0
4 0 0 0
5 0 0 0
6 0 0 0
7 0 0 0
8 0 0 0
9 0 0 0
2 5 0 k 1 M 1 0 M / 5 M
D B s i z e
J E D S
C D S
![Page 42: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/42.jpg)
Authentication Performance• AMD 4-processor dual-core• 10 million entry DB
1 0 M
0
5 0 0 0
1 0 0 0 0
1 5 0 0 0
2 0 0 0 0
1 2 3 4 5 6 7 8 9 1 0 1 1
t i m e
Au
ths/
seco
nd
C o l d c a c h e
W a r m c a c h e
![Page 43: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/43.jpg)
BDB 4.2 performance
StartSingle
SingleSingle
SingleFour
FourFour
Four
0
250000
500000
750000
1000000
1250000
1500000
1750000
2000000
2250000
00:00.00
00:17.28
00:34.56
00:51.84
01:09.12
01:26.40
01:43.68
02:00.96
02:18.24
02:35.52
02:52.80
Malloc performance
Glibc size
Glibc time
Hoard size
Hoard time
Umem size
Umem time
Tcmalloc size
Tcmalloc timePro
cess
siz
e, K
B
![Page 44: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/44.jpg)
BDB 4.5 Performance
StartSingle
SingleSingle
FourFour
FourFour
0
200000
400000
600000
800000
1000000
1200000
1400000
1600000
1800000
2000000
00:00.00
00:21.60
00:43.20
01:04.80
01:26.40
01:48.00
02:09.60
02:31.20
02:52.80
03:14.40
Malloc performance
Glibc size
Glibc time
Hoard size
Hoard time
Umem size
Umem time
Tcmalloc size
Tcmalloc timePro
cess
siz
e, K
B
![Page 45: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/45.jpg)
BDB 4.6 Performance
StartSingle
SingleSingle
FourFour
FourFour
0
250000
500000
750000
1000000
1250000
1500000
1750000
2000000
2250000
00:00.00
00:08.64
00:17.28
00:25.92
00:34.56
00:43.20
00:51.84
01:00.48
01:09.12
01:17.76
01:26.40
01:35.04
01:43.68
01:52.32
Malloc performance
Glibc size
Glibc time
Hoard size
Hoard time
Umem size
Umem time
Tcmalloc size
Tcmalloc timePro
cess
siz
e, K
B
![Page 46: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/46.jpg)
Worst Case Search Performance
StartSingle
SingleSingle
FourFour
FourFour
0
250000
500000
750000
1000000
1250000
1500000
1750000
2000000
2250000
2500000
2750000
00:00.00
00:43.20
01:26.40
02:09.60
02:52.80
03:36.00
04:19.20
05:02.40
05:45.60
06:28.80
07:12.00
07:55.20
08:38.40
Search performance
2.0.27 size
2.0.27 time
2.1.30 size
2.1.30 time
2.2.30 size
2.2.30 time
2.3.33 size
2.3.33 time2.4 size
2.4 time
Pro
cess
siz
e, K
B
![Page 47: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/47.jpg)
Cached Search Performance
StartSingle
SingleSingle
FourFour
FourFour
0250000500000750000
1000000125000015000001750000200000022500002500000275000030000003250000350000037500004000000
00:00.00
00:08.64
00:17.28
00:25.92
00:34.56
00:43.20
00:51.84
01:00.48
01:09.12
01:17.76
01:26.40
01:35.04
01:43.68
Cached Search performance
2.0.27 size
2.0.27 time
2.1.30 size
2.1.30 time
2.2.30 size
2.2.30 time
2.3.33 size
2.3.33 time2.4 size
2.4 time
Pro
cess
siz
e, K
B
![Page 48: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/48.jpg)
Current Performance
StartSingle
SingleSingle
FourFour
FourFour
0
200000
400000
600000
800000
1000000
1200000
1400000
1600000
1800000
2000000
00:00.00
00:04.3200:08.64
00:12.9600:17.28
00:21.60
00:25.9200:30.24
00:34.5600:38.88
00:43.2000:47.52
00:51.8400:56.16
01:00.4801:04.80
Malloc performance
Glibc size
Glibc time
Hoard size
Hoard timeUmem size
Umem time
Tcmalloc size
Tcmalloc timePro
cess
siz
e, K
B
![Page 49: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/49.jpg)
Database Parameters• 380836 entries
– Range in size from 3K to 10MB
• Total size on disk ~1.3GB• Running on Socket939 2.4GHz AMD64 X2
w/512KB L2 cache per core, 4GB DDR400 ECC/REG RAM
• No disk I/O during searches• Using 2.3 as of November 2006 unless
otherwise noted
![Page 50: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/50.jpg)
2.0.27 DB Parameters
Ldbm BDB 4.2.52 dbnosync, dbcachesize 512MB
slapadd 113.455u 8.004s 2:16.96 88.6% 0+0k 0+0io 0pf+0w
total 1281133
-rw------- 1 hyc users 88879104 2007-02-09 23:46 dn2id.dbb
-rw------- 1 hyc users 1220915200 2007-02-09 23:46 id2entry.dbb
-rw------- 1 hyc users 8192 2007-02-09 23:46 nextid.dbb
-rw------- 1 hyc users 798720 2007-02-09 23:46 objectClass.dbb
![Page 51: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/51.jpg)
2.1.30 DB Parametersbdb BDB 4.2.52 TXN_NOSYNC, TXN_NOT_DURABLE
slapadd 162.582u 8.300s 3:04.30 92.7% 0+0k 0+0io 7189pf+0w
total 850295
-rw------- 1 hyc users 16384 2007-02-10 04:35 __db.001
-rw------- 1 hyc users 536870912 2007-02-10 04:35 __db.002
-rw------- 1 hyc users 2359296 2007-02-10 04:35 __db.003
-rw------- 1 hyc users 663552 2007-02-10 04:35 __db.004
-rw------- 1 hyc users 16384 2007-02-10 04:35 __db.005
-rw-r--r-- 1 hyc users 177 2007-02-10 01:30 DB_CONFIG
-rw------- 1 hyc users 79978496 2007-02-10 04:38 dn2id.bdb
-rw------- 1 hyc users 782745600 2007-02-10 04:38 id2entry.bdb
-rw------- 1 hyc users 28 2007-02-10 04:35 log.0000000001
-rw------- 1 hyc users 6553600 2007-02-10 04:38 objectClass.bdb
![Page 52: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/52.jpg)
2.2.30 DB Parametersbdb BDB 4.2.52 TXN_NOSYNC, TXN_NOT_DURABLE
slapadd 284.789u 10.836s 5:04.65 97.0% 0+0k 0+0io 7136pf+0w
total 1869554
-rw------- 1 hyc users 16384 2007-02-10 02:42 __db.001
-rw------- 1 hyc users 536870912 2007-02-10 02:42 __db.002
-rw------- 1 hyc users 2359296 2007-02-10 02:42 __db.003
-rw------- 1 hyc users 663552 2007-02-10 02:42 __db.004
-rw------- 1 hyc users 32768 2007-02-10 02:42 __db.005
-rw-r--r-- 1 hyc users 177 2007-02-10 01:30 DB_CONFIG
-rw------- 1 hyc users 79978496 2007-02-10 02:47 dn2id.bdb
-rw------- 1 hyc users 1288781824 2007-02-10 02:47 id2entry.bdb
-rw------- 1 hyc users 28 2007-02-10 02:47 log.0000000001
-rw------- 1 hyc users 6549504 2007-02-10 02:47 objectClass.bdb
![Page 53: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/53.jpg)
2.3.33 DB Parametersbdb BDB 4.2.52 TXN_NOSYNC, TXN_NOT_DURABLE
slapadd 118.447u 9.256s 2:17.26 93.0% 0+0k 0+0io 7126pf+0w
total 1344299
-rw-r--r-- 1 hyc users 2048 2007-02-10 04:42 alock
-rw------- 1 hyc users 16384 2007-02-10 04:40 __db.001
-rw------- 1 hyc users 536870912 2007-02-10 04:40 __db.002
-rw-r--r-- 1 hyc users 177 2007-02-10 01:30 DB_CONFIG
-rw------- 1 hyc users 79978496 2007-02-10 04:42 dn2id.bdb
-rw------- 1 hyc users 1288142848 2007-02-10 04:42 id2entry.bdb
-rw------- 1 hyc users 6549504 2007-02-10 04:42 objectClass.bdb
![Page 54: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/54.jpg)
2.4 DB Parametersbdb BDB 4.2.52 TXN_NOSYNC, TXN_NOT_DURABLE
slapadd 138.416u 10.060s 2:39.27 93.2% 0+0k 0+0io 7127pf+0w
total 1348137
-rw-r--r-- 1 hyc users 2048 2007-02-10 05:39 alock
-rw------- 1 hyc users 16384 2007-02-10 05:39 __db.001
-rw------- 1 hyc users 536870912 2007-02-10 05:39 __db.002
-rw------- 1 hyc users 2359296 2007-02-10 05:39 __db.003
-rw------- 1 hyc users 663552 2007-02-10 05:39 __db.004
-rw------- 1 hyc users 32768 2007-02-10 05:39 __db.005
-rw-r--r-- 1 hyc users 177 2007-02-10 01:30 DB_CONFIG
-rw------- 1 hyc users 79978496 2007-02-10 05:38 dn2id.bdb
-rw------- 1 hyc users 1292042240 2007-02-10 05:38 id2entry.bdb
-rw------- 1 hyc users 6549504 2007-02-10 05:38 objectClass.bdb
![Page 55: OpenLDAP - Highland Sun · – Component-based matching – More overlays – Dynamic reconfiguration. OpenLDAP Today • The fastest Directory Server • The most reliable • The](https://reader034.vdocuments.us/reader034/viewer/2022050201/5f54d309cd0e02554d7e52bd/html5/thumbnails/55.jpg)
ldapadd performance• DB taking 2:42.64 for slapadd -q took
1:33:08.74 using ldapadd• Optimized server and client in HEAD bring
the time down to 5:20.00• Remaining network and encode/decode
overhead unlikely to go away