benchmark testing results: csc’s integral life insurance ... · system, sql server provides a...
TRANSCRIPT
Benchmark Testing Results: CSC’s Integral Life Insurance Solution and Microsoft SQL Server i
Benchmark Testing Results: CSC’s Integral Life
Insurance Software Running on SQL Server
Running CSC’s Integral life insurance solution on Microsoft SQL Server and Windows Server provides
unmatched performance and scalability
Technical White Paper Published: January 2013 Applies to: Microsoft SQL Server 2012
Abstract
In June through August of 2012, CSC and Microsoft conducted performance and scalability testing on CSC’s
Integral1 life insurance software running on Microsoft SQL Server 2012 Enterprise data management software
and the Windows Server 2008 R2 Enterprise operating system.
The benchmark testing was very successful, demonstrating a near-linear scalability to 1,500 concurrent users
running against a database of 1 million policies while maintaining excellent performance. These results
confirm that SQL Server is an excellent choice for CSC’s Integral licensees.
1 Integral is a trademark of CSC, registered in many jurisdictions worldwide.
Benchmark Testing Results: CSC’s Integral Life Insurance Solution and Microsoft SQL Server ii
©2013 Microsoft Corporation. All rights reserved. This document is provided “as-is.” Information and views
expressed in this document, including URL and other Internet Web site references, may change without notice.
You bear the risk of using it.
This document does not provide you with any legal rights to any intellectual property in any Microsoft product.
You may copy and use this document for your internal, reference purposes.
Benchmark Testing Results: CSC’s Integral Life Insurance Solution and Microsoft SQL Server iii
Table of Contents
Introduction............................................................................................................................................... 1
Introducing CSC’s Integral ................................................................................................................................ 1
Microsoft and CSC Team Up for Insurance ....................................................................................................... 2
Benefits of SQL Server....................................................................................................................................... 3
Benchmark Testing Overview ..................................................................................................................... 4
Test Environment.............................................................................................................................................. 5 Application and Load Generation Servers .................................................................................................... 6 Database Server ........................................................................................................................................... 6 Storage ......................................................................................................................................................... 7
Test Scenarios ................................................................................................................................................... 7
Benchmark Testing Results......................................................................................................................... 8
Single Application Server Tests ......................................................................................................................... 8 Application Server Metric for Single Server Tests ...................................................................................... 10 Database Metrics for Single Server Tests ................................................................................................... 11
Multiple Application Server Tests ................................................................................................................... 12 Application Server Metrics for Multiple Server Tests ................................................................................ 15 Database Server Metrics for Multiple Server Tests .................................................................................... 16
Summary ................................................................................................................................................. 17
Additional Information ............................................................................................................................ 18
About CSC ....................................................................................................................................................... 18
About Microsoft ............................................................................................................................................. 18
Benchmark Testing Results: CSC’s Integral Life Insurance Solution and Microsoft SQL Server 1
“We are thrilled to see insurance carriers increasingly deploy mission-critical applications on the Microsoft platform. CSC’s commitment to offering Integral on Windows Server and SQL Server will provide a more modern, configurable core system with the performance and stability that insurers require and a TCO that is second to none.”
Karen T. Cone General Manager
Worldwide Financial Services Microsoft Corporation
“To give insurers more choices, we have enhanced some of our most widely used software for platform portability and reuse globally. CSC remains committed to delivering the right solutions to the right markets at the right time on a choice of platforms to help our clients boost customer service, create efficiencies, and meet their growth objectives.”
Ray August President
Financial Services Group CSC
Introduction The flattening of the global market provides a world of
opportunities and challenges for insurers. Companies need
flexible, dependable technology that helps them respond
quickly to changing conditions and meet the local needs of
diverse markets.
CSC’s Integral insurance software suite helps insurance carriers
meet these challenges with a comprehensive solution that is
ideally suited for many scenarios, such as:
Multi-national firms seeking a single platform for regional or international operations.
Domestic insurance firms wanting to launch new products quickly without disrupting their
existing processing.
Insurers needing a self-contained insurance administration solution for quick start-up of
new ventures.
Multi-line insurers looking for a complete, low-maintenance processing capability for a
secondary line of business.
The underlying core of an insurance solution is the data collected;
this data is maintained in an enterprise database such as
Microsoft SQL Server. Running on the Windows Server operating
system, SQL Server provides a reliable, high performing, and
scalable foundation for CSC’s Integral software.
This white paper begins with an introduction to CSC’s Integral
insurance software suite. Next, the paper provides an overview
of SQL Server, a proven database platform for enterprise
deployments of Integral. The many advantages SQL Server brings
to Integral software are highlighted, including manageability,
scalability, reliability, and affordability.
The white paper then presents benchmark test results that confirm that SQL Server scales to meet
the needs of even the largest deployments of CSC’s Integral life insurance software. Enterprise
customers can be confident in the choice of SQL Server and Windows Server as a foundation for
CSC’s Integral life insurance solution.
Introducing CSC’s Integral
CSC’s Integral is a comprehensive administration suite for life insurance and annuities/pensions,
property and casualty (P&C)/general insurance, and group insurance that provides end-to-end
policy/group management, product configuration, and business-analytics capabilities. Its portable
Benchmark Testing Results: CSC’s Integral Life Insurance Solution and Microsoft SQL Server 2
architecture lets companies implement Integral quickly and operate cost-effectively, either in-house
or through CSC’s business process services.
For more information about Integral, visit www.csc.com.
Microsoft and CSC Team Up for Insurance
Microsoft and CSC have formed an alliance to support CSC’s Integral on Microsoft software.
Because Integral interoperates seamlessly with Windows Server and SQL Server, insurance carriers
who deploy the joint CSC/Microsoft solution obtain a modern, easy-to-manage solution for
improving processes, customer service, and overall productivity. The Windows Server operating
system delivers the power and reliability that businesses require to thrive and grow, while SQL
Server provides the performance, availability, and security to manage large volumes of data
intelligently and eliminate silos of information. The result is a streamlined solution with global reach
that lets carriers operate regionally and internationally with ease.
Figure 1 shows CSC’s Integral on Microsoft software.
Figure 1. CSC's Integral on Microsoft software
For more information about how CSC and Microsoft are working together, visit
http://www.microsoft.com/enterprise/partners/csc.aspx.
Benchmark Testing Results: CSC’s Integral Life Insurance Solution and Microsoft SQL Server 3
Benefits of SQL Server
SQL Server is a comprehensive, integrated, and enterprise-ready data-management software
solution. It provides a reliable, cost-effective, low-maintenance database framework for CSC’s
Integral that can support the largest and most process-intensive deployments. Hundreds of
enterprises are currently running 10 terabyte and larger transactional databases on SQL Server.
Running CSC’s Integral on SQL Server provides customers with many benefits:
Scalability and performance features
SQL Server includes features that can help CSC customers scale and maintain performance
as their businesses grow.
Six nines (99.9999 percent) uptime availability
CSC customers can protect their mission-critical databases from downtime and data loss
with six nines uptime availability.2,3
Faster to deploy
On average, SQL Server database administrators (DBAs) can install and configure new
database servers in 1.5 hours, while the largest competitor’s DBAs can take 6 hours.4
Lower hardware cost
SQL Server can run on standard commodity server hardware, which can dramatically lower
the total cost of ownership (TCO) for customers.
Lower software costs
The list price of SQL Server is a third of the largest competitors;5 in addition, SQL Server
includes major database-related features, such as high availability; remote disaster
recovery; partitioning; data compression; transparent data encryption; spatial; master data
management; complex event processing; extract, transform, and load (ETL); online
analytical processing (OLAP); data mining; reporting services; and self-service business
intelligence (BI) tools. Competitors’ licensing models add costs for options and add-ins.6
Simpler systems management and lower staffing costs
SQL Server database administrators can typically manage four times as many physical
databases as a competitor’s DBAs, leading to an estimated annual savings of $5,779 in
administrative costs per database, a 460 percent difference in annual cost of administration
per database.7
2 http://www.microsoft.com/sqlserver/en/us/product-info/why-sql-server.aspx 3 http://www.microsoft.com/casestudies/Microsoft-SQL-Server-2008-R2-Enterprise/Stratus-Technologies/Protect-your-mission-critical-databases-from- downtime-and-data-loss-with-six-nines-uptime-availability/4000007136 4 http://www.alinean.com/PDFs/Microsoft_SQL_Server_and_Oracle-Alinean_TCA_Study_2010.pdf 5 http://www.microsoft.com/sqlserver/en/us/tools/cost-savings-calclator.aspx 6 http://www.microsoft.com/sqlserver/en/us/product-info/competitor-compare.aspx 7 http://www.microsoft.com/sqlserver/en/us/product-info/why-sql-server.aspx
Benchmark Testing Results: CSC’s Integral Life Insurance Solution and Microsoft SQL Server 4
Fewer security vulnerabilities
Since 2002, SQL Server has recorded the fewest reported vulnerabilities of any of the major
database platforms as compiled by the National Institute of Standards and Technology
(NIST).8
With SQL Server, CSC customers can save with reduced licensing, hardware, administration, and
support fees, which translate into substantially lower costs over the life of the system.
For more information about SQL Server, visit www.microsoft.com/sql.
Benchmark Testing Overview As part of their strategic alliance, CSC and Microsoft ran real-world benchmarking tests at the
Microsoft Platform Adoption Center (PAC) in Redmond, Washington, to evaluate and optimize the
performance and scalability of CSC’s Integral life insurance component running on SQL Server 2012
and Windows Server 2008 R2 Enterprise.
The benchmarking tests aimed to simulate the typical daily activities of agents at a large insurer,
with up to 1,500 concurrent users running against a database of 1 million policies. This was
achieved on five application servers and a database server, all industry-standard with a total list
price of less than $64,000,9 and industry-standard storage with a list price of $140,00010—a
compelling scenario for customers.
A variety of tests were performed, including:
Single server tests
These tests investigated the performance of Integral on a single application server with
various numbers of virtual users. Note that based on the test methodology, 125 virtual
users are equivalent to 165 actual concurrent users.
Multiple server tests
These tests looked at the horizontal scalability of the solution by starting with one
application server with 200 concurrent users, and adding application servers, each with 200
concurrent users, up to five application servers with 1,000 concurrent users. A test was then
run with five application servers and 1,500 concurrent users, representing a very large
insurance company.
8 http://itic-corp.com/blog/2010/09/sql-server-most-secure-database-oracle-least-secure-database-since-2002 9 Manufacturer's suggested retail price (MSRP) in April 2012 10 Manufacturer's suggested retail price (MSRP) in April 2012
Benchmark Testing Results: CSC’s Integral Life Insurance Solution and Microsoft SQL Server 5
Test Environment
The test infrastructure included five major components:
Database server
Application servers
Load-generation servers
Domain controller/Active Directory Domain Services
Storage
Details of the servers used are shown in Table 1.
Table 1. Servers used in benchmark testing
Server Number
of servers Make/model Cache
Number of cores
CPU RAM
Database server 1 Intel four socket 24 MB 32
Four Intel Xeon
X7550 processors
(2.0 GHz)
128 GB
Application servers 5 HP ProLiant
DL380 G7 12 MB 12
Two Intel Xeon
X5650 processors
(2.80 GHz)
64 GB
Load-generation servers
3 HP ProLiant
DL380 G7 12 MB 12
Two Intel Xeon
X5650 processors
(2.80 GHz)
64 GB
1 IBM System
x3850 X5 24 MB 32
Four Intel Xeon
X7560 processors
(2.26 GHz)
256 GB
Domain controller/ Active Directory Domain Services
1 Dell PowerEdge
1850 2 MB 2
Two Intel Xeon
processors
(3.40 GHz)
4 GB
Storage X-IO Hyper ISE
Two Datapacs (10 solid-state drives [SSD] and 10 serial-
attached SCSI [SAS] drives each), capable of 30K-60K
IOPS
Figure 2 shows a schematic of the infrastructure used in the benchmark testing.
Benchmark Testing Results: CSC’s Integral Life Insurance Solution and Microsoft SQL Server 6
HPProLiant
DL380 G6
FANS
PROC
1
PROC
2
POWER
SUPPLY
2POWER
SUPPLY
1OVERTEMP
POWERCAP
1 2 3 4
9
8
7
6
5
4
3
2
1 1
2
3
4
5
6
7
8
9
ONLINESPARE
MIRROR
UID
2
1
4
3
6
5
8
76 5 4 3 2 1
HP ProLiant DL380G712 x 2.80 GHz—64 GB RAM
Back-End Tier(Database)
Windows Server 2008 R2 Enterprise with SP1
ISE
Front-End Tier(Application/Load Generation)
Windows Server 2008 R2 Enterprise with SP1
PowerEdge 1950
0
1
2
3
Dell PowerEdge 18502 x 3.40 GHz—4 GB RAM
Active Directory Domain Services/Domain Name System
Windows Server 2008 R2 EnterpriseESD
0
3
1
4
2
5
System x3850
IBM System x3850 X532 x 2.26 GHz—256 GB RAM
X-IO storageHyper ISE
Pool 2 10 x 10K SAS
10 x SSDHP
ProLiantDL380 G6
FANS
PROC
1
PROC
2
POWER
SUPPLY
2POWER
SUPPLY
1OVERTEMP
POWERCAP
1 2 3 4
9
8
7
6
5
4
3
2
1 1
2
3
4
5
6
7
8
9
ONLINESPARE
MIRROR
UID
2
1
4
3
6
5
8
76 5 4 3 2 1
Intel Xeon 4-socket32 x 2.0 GHz—128 GB RAM
SQL Server 2012 Enterprise
HP ProLiant DL380G712 x 2.80 GHz—64 GB RAM
HPProLiant
DL380 G6
FANS
PROC
1
PROC
2
POWER
SUPPLY
2POWER
SUPPLY
1OVERTEMP
POWERCAP
1 2 3 4
9
8
7
6
5
4
3
2
1 1
2
3
4
5
6
7
8
9
ONLINESPARE
MIRROR
UID
2
1
4
3
6
5
8
76 5 4 3 2 1
HP ProLiant DL380G712 x 2.80 GHz—64 GB RAM
HP ProLiant DL380G712 x 2.80 GHz—64 GB RAM
HPProLiant
DL380 G6
FANS
PROC
1
PROC
2
POWER
SUPPLY
2POWER
SUPPLY
1OVERTEMP
POWERCAP
1 2 3 4
9
8
7
6
5
4
3
2
1 1
2
3
4
5
6
7
8
9
ONLINESPARE
MIRROR
UID
2
1
4
3
6
5
8
76 5 4 3 2 1
HPProLiant
DL380 G6
FANS
PROC
1
PROC
2
POWER
SUPPLY
2POWER
SUPPLY
1OVERTEMP
POWERCAP
1 2 3 4
9
8
7
6
5
4
3
2
1 1
2
3
4
5
6
7
8
9
ONLINESPARE
MIRROR
UID
2
1
4
3
6
5
8
76 5 4 3 2 1
HP ProLiant DL380G712 x 2.80 GHz—64 GB RAM
HPProLiant
DL380 G6
FANS
PROC
1
PROC
2
POWER
SUPPLY
2POWER
SUPPLY
1OVERTEMP
POWERCAP
1 2 3 4
9
8
7
6
5
4
3
2
1 1
2
3
4
5
6
7
8
9
ONLINESPARE
MIRROR
UID
2
1
4
3
6
5
8
76 5 4 3 2 1
HP ProLiant DL380G712 x 2.80 GHz—64 GB RAM
HPProLiant
DL380 G6
FANS
PROC
1
PROC
2
POWER
SUPPLY
2POWER
SUPPLY
1OVERTEMP
POWERCAP
1 2 3 4
9
8
7
6
5
4
3
2
1 1
2
3
4
5
6
7
8
9
ONLINESPARE
MIRROR
UID
2
1
4
3
6
5
8
76 5 4 3 2 1
HP ProLiant DL380G712 x 2.80 GHz—64 GB RAM
SQL2 DBDrive E600 GB
8765432187654321
UID21
G 5
HPProLiantDL580
Ap
plicatio
n servers
Load
-generatio
n servers
HPProLiant
DL380 G6
FANS
PROC
1
PROC
2
POWER
SUPPLY
2POWER
SUPPLY
1OVERTEMP
POWERCAP
1 2 3 4
9
8
7
6
5
4
3
2
1 1
2
3
4
5
6
7
8
9
ONLINESPARE
MIRROR
UID
2
1
4
3
6
5
8
76 5 4 3 2 1
HP ProLiant DL380G712 x 2.80 GHz—64 GB RAM
SQL1 TempDBDrive F500 GB
SQL1 LogsDrive G400 GB
SQL2 ScratchDrive S
1 TB
Pool 1 10 x 10K SAS
10 x SSD
SQL1 DBDrive E600 GB
SQL2 TempDBDrive F500 GB
SQL2 LogsDrive G400 GB
SQL1 ScratchDrive S
1 TBD
link D
SX
-33
50S
R 1
Gb
ps E
the
rne
t
Figure 2. Benchmark testing equipment
Application and Load Generation Servers
The tests used up to five Hewlett-Packard (HP) ProLiant DL380 G7 application servers, and one IBM
System x3850 X5 and up to three HP ProLiant DL380 G7 load-generation servers. This ensured that
the tests were carried out at full throttle, and that the tests provided an accurate measurement of
the scalability of the system.
Database Server
A single Intel four-socket database server was used, taking advantage of the latest generation of the
Intel Xeon processor E7 family.
The configuration was as follows:
SQL Server 2012 Enterprise
Windows Server 2008 R2 Enterprise with Service Pack 1 (SP1)
Four Intel Xeon X7550 processors, for a total of 32 cores
128 GB of random access memory (RAM)
Benchmark Testing Results: CSC’s Integral Life Insurance Solution and Microsoft SQL Server 7
Storage
X-IO Hyper ISE storage modules provided 19.5 terabytes of storage for the database tables and
indices. However, less than 1 terabyte of the total capacity of the storage was used for the tests.
Test Scenarios
The benchmark testing used 16 automated test scenarios that represent more than 90 percent of
the transaction types that are performed daily at a typical insurance firm, including inquiries,
modifications, and additions. Each test scenario involved between three and ten user steps, with a
“think time” of 10 seconds between steps. Completion times varied between test scenarios.
The 16 tests scenarios and the number of steps in each are listed in Table 2.
Table 2. Test scenarios
Test number Test scenarios Number of steps
Test 1 Add personal client 3
Test 2 Modify bank/branch 3
Test 3 Group member inquiry 5
Test 4 Agent appointment 3
Test 5 New contract/universal variable life 10
Test 6 New contract/hospital cash plan 8
Test 7 New contract/flexible premium 8
Test 8 Medical provider inquiry 5
Test 9 Policy load inquiry 3
Test 10 Modify allocation rules 3
Test 11 Table inquiry 6
Test 12 Help lines inquiry 3
Test 13 Policy notes inquiry 3
Test 14 Reassurer inquiry 5
Test 15 Modify client bank account 3
Test 16 Life Insurance Association records inquiry 5
For the benchmarking tests, the average throughput of a mix of the 16 test scenarios was
measured.
Benchmark Testing Results: CSC’s Integral Life Insurance Solution and Microsoft SQL Server 8
As an example, Table 3 lists the steps in three typical test scenarios: New contract (NC)—universal
variable life (test 5), New contract—hospital cash plan (test 6), and Table inquiry (test 11). Note that
some steps are performed more than once.
Table 3. Examples of steps in test scenarios
New contract—universal
variable life
New contract—hospital cash
plan Table inquiry
1. New contract proposal sub-
menu
1. New contract proposal sub-
menu
1. Tables and codes
maintenance submenu
2. Universal variable life 2. Hospital cash plan 2. Table item scroll inquiry
3. Lives assured 3. Lives assured 3. Table item maintenance
4. Underwriting questionnaire 4. Underwriting questionnaire 4. Alter/cancel from inception
subroutine
5. Lives assured 5. Lives assured 5. Table item scroll inquiry
6. Select cover required 6. Select cover required 6. Tables and codes
maintenance submenu
7. Universal variable life 7. Work with proposal
8. Premium direction 8. New contract proposal
submenu
9. Work with proposal
10. New contract proposal
submenu
Benchmark Testing Results The following sections provide detailed results of the benchmark tests.
Single Application Server Tests
The team from CSC and Microsoft performed a series of single application server benchmarking
tests with the goal of identifying the optimal number of users that can be served by a single entry-
level application server.
These tests were performed on one application server (HP ProLiant DL380 G7 with 12 cores and 64
GB RAM) and one database server (Intel Xeon four-socket), using the IBM System x3850 X5 server
to generate the test workload.
Benchmark Testing Results: CSC’s Integral Life Insurance Solution and Microsoft SQL Server 9
For these single application server tests, the team added virtual users up to a maximum of 250
virtual users, which is equivalent to 330 concurrent users. The team ran each scenario several
times, and calculated an average of these test runs.
Table 4 lists the response times, or average time to complete each test scenario, for the various
numbers of virtual users on a single server.
Table 4. Results of single application server tests with various numbers of concurrent users
Test scenarios Response time (seconds)
165 users 200 users 230 users 266 users 330 users
Add personal client 1.31 1.39 1.47 1.5 1.67
Modify bank/branch 0.98 1.01 1.10 1.18 1.38
Group member inquiry 2.17 2.02 2.59 2.42 2.7
Agent appointment 1.01 1.09 1.18 1.23 1.3
NC/universal variable life 3.65 3.61 4.07 4.22 4.85
NC/hospital cash plan 2.6 2.65 3.00 3.05 3.46
NC/flexible premium 2.49 2.65 2.81 2.92 3.44
Medical provider inquiry 1.28 1.32 1.51 1.58 1.74
Policy load inquiry 0.78 0.76 0.87 0.9 1.04
Modify allocation rules 0.78 0.82 0.91 0.89 1.08
Table inquiry 1.18 1.25 1.34 1.38 1.62
Help lines inquiry 0.83 0.83 0.99 0.96 1.19
Policy notes inquiry 1.00 0.99 1.14 1.12 1.33
Reassurer inquiry 1.07 1.11 1.16 1.22 1.39
Modify client bank
account 0.83 0.90 0.95 0.97 1.09
Life Insurance
Association records
inquiry
0.99 1.01 1.15 1.17 1.48
Benchmark Testing Results: CSC’s Integral Life Insurance Solution and Microsoft SQL Server 10
Figure 3 visualizes the transaction processing times for the test scenarios for a single server with
varying numbers of virtual users.
Figure 3. Single server user test results, transaction processing time per test scenario
Application Server Metric for Single Server Tests
Table 5 shows total throughput and the average and maximum application server memory and CPU
utilizations in the single server tests.
Table 5. Application server metrics for single server tests
Application server metrics 165
users 200
users 230
users 266
users 330
users
Average application server memory utilization 7.3% 8.0% 8.5% 9.4% 10.8%
Maximum application server memory
utilization 8.0% 8.7% 9.0% 10.3% 11.7%
Average application server CPU utilization 21.1% 25.6% 32.8% 37.6% 45.5%
Maximum application server CPU utilization 40.7% 39.0% 58.0% 69.0% 77.0%
Total throughput (business transactions per
minute) 489 565 638 707 811
Figure 4 visualizes the application server CPU and memory utilization and the total number of
transactions. Note that the average application server CPU utilization remained below 50 percent,
which is considered acceptable.
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
4.50
5.00
Tran
sact
ion
pro
cess
ing
tim
e (s
eco
nd
s)
Add personal client
Modify bank/branch
Group member inquiry
Agent appointment
NC-universal variable life
NC-hospital cash plan
NC-flexible premium
Medical provider inquiry
Policy loan inquiry
Modify allocation rules
Table inquire
Help lines inquiry
Policy notes inquiry
Reassurer inquiry
Modify client bank account
LIA records inquiry165 200 230 266 330 Number of concurrent users
Test scenarios
Benchmark Testing Results: CSC’s Integral Life Insurance Solution and Microsoft SQL Server 11
Figure 4. Single server total number of transactions and CPU and memory utilizations
Figure 3 and Figure 4 show that there is a “sweet spot” between 200 and 230 concurrent users in
terms of throughput, execution time, and CPU utilization.
Database Metrics for Single Server Tests
Table 6 shows the CPU utilization by SQL Server and the number of batch requests per second
processed by SQL Server in the single server tests. Note that the average database server CPU
utilization remained below 1 percent. The average number of batch requests per second processed
by SQL Server was 2,136 with 330 concurrent users.
Table 6. Database server metrics for single server tests
Database server metrics 165
users 200
users 230
users 266
users 330
users
Average database server CPU utilization 0.5% 0.6% 0.8% 0.8% 0.9%
Maximum database server CPU utilization 0.7% 0.7% 0.9% 1.0% 1.1%
Average number of batch requests per
second 1,102 1,319 1,524 1,735 2,136
Maximum number of batch requests per
second 1,269 1,583 1,680 1,923 2,309
0
100
200
300
400
500
600
700
800
900
0
10
20
30
40
50
60
70
80
90 Average CPUutilization (%)
Maximum CPUutilization (%)
Average memoryutilization (%)
Maximum memoryutilization (%)
Total throughput(businesstransactions/minute)
165 200 230 266 330
Number of concurrent users
Per
cen
t u
tiliz
atio
n
Tota
l th
rou
ghp
ut
(bu
sin
ess
tran
sact
ion
s/m
inu
te)
Benchmark Testing Results: CSC’s Integral Life Insurance Solution and Microsoft SQL Server 12
Figure 5 visualizes the database server metrics for the single server tests.
Figure 5. CPU utilization by SQL Server and the number of batch requests per second processed by SQL Server in the single users tests
Multiple Application Server Tests
To confirm the scalability of the solution, the CSC and Microsoft team performed benchmark tests
using multiple application servers in a scale-out configuration. These tests simulated 200 concurrent
users (150 virtual users) per application server and added application servers up to a total of five
with 1,000 concurrent users. A final test was run with five application servers and 1,500 concurrent
users. The team ran each scenario several times and calculated an average of these test runs.
The test results showed that as the number of servers increased from one to five―and as the
corresponding number of concurrent users increased from 200 to 1,000―the throughput increased
nearly linearly.
At the same time, the average number of transactions per server remained constant when adding
application servers up to five application serves and 1,000 users; however, the average number of
transactions per server increased for the final test that used five applications servers and 1,500
concurrent users. These results show the effectiveness of using a scale-out deployment to increase
the overall throughput.
0
500
1000
1500
2000
2500
0
0.2
0.4
0.6
0.8
1
1.2Average CPUutilization (%)
Maximum CPUutilization (%)
Average number ofbatch requests/sec
Maximum numberof batchrequests/sec
165 200 230 266 330
Nu
mb
er o
f b
atch
req
ues
ts/s
eco
nd
Per
cen
t u
tiliz
atio
n
Number of concurrent users
Benchmark Testing Results: CSC’s Integral Life Insurance Solution and Microsoft SQL Server 13
Table 7 lists the response times, or average time to complete each test scenario, for the various
numbers of servers and users in the multiple user tests.
Table 7. Results of multiple application server tests with various numbers of simulated concurrent users
Test scenarios
Response time (seconds)
One app server (200
users)
Two app servers
(400 users)
Three app servers
(600 users)
Four app servers
(800 users)
Five app servers (1,000 users)
Five app servers (1,500 users)
Add personal client 1.31 1.34 1.28 1.31 1.28 1.24
Modify bank/branch 1.08 1.12 1.02 1.02 1.01 1.05
Group member inquiry 2.06 2.28 2.19 2.17 2.03 2.07
Agent appointment 1.06 1.07 1.03 1.01 1.03 1.09
NC/universal variable life 3.76 3.71 3.63 3.95 3.70 4.09
NC/hospital cash plan 2.74 2.68 2.64 2.65 2.68 2.91
NC/flexible premium 2.66 2.61 2.56 2.58 2.57 2.83
Medical provider inquiry 1.36 1.38 1.28 1.28 1.30 1.39
Policy load inquiry 0.85 0.82 0.79 0.78 0.77 0.81
Modify allocation rules 0.87 0.82 0.74 0.75 0.76 0.77
Table inquiry 1.29 1.21 1.18 1.20 1.20 1.26
Help lines inquiry 0.58 0.86 0.82 0.83 0.81 0.87
Policy notes inquiry 1.04 1.06 1.02 0.99 0.96 1.02
Reassurer inquiry 1.10 1.09 1.02 1.01 1.04 1.09
Modify client bank account 0.90 0.86 0.83 0.81 0.81 0.88
Life Insurance Association
records inquiry 1.13 1.06 1.02 1.03 1.00 1.07
Benchmark Testing Results: CSC’s Integral Life Insurance Solution and Microsoft SQL Server 14
Figure 6 visualizes the response times for the various test scenarios.
Figure 6. Multiple server test results, transaction processing time per test scenario
Table 8 shows total throughput and the average and maximum application server memory and CPU
utilizations in the multiple server tests.
Table 8. Application server metrics for multiple server tests
Application server metrics
One app
server (200
users)
Two app servers
(400 users)
Three app
servers (600
users)
Four app servers
(800 users)
Five app servers (1,000 users)
Five app servers (1,500 users)
Average application server
memory utilization 6.8% 6.6% 6.47% 6.6% 6.5% 7.0%
Maximum application server
memory utilization 7.3% 7.5% 7.2% 7.3% 7.2% 7.9%
Average application server CPU
utilization 27.7% 29.9% 26.8% 26.6% 27.8% 38.8%
Maximum application server
CPU utilization 43.0% 47.3% 43.0% 41.0% 43.8% 62.9%
Average number of transactions
per server, per minute 670 680 680 677 680 890
Total throughput (business
transactions per minute) 670 1,306 2,040 2,708 3,400 4,450
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
1 server,200 users
2 servers,400 users
3 servers,600 users
4 servers,800 users
5 servers,1,000 users
5 servers,1,500 users
Tran
sact
ion
pro
cess
ing
tim
e (s
eco
nd
s)
Add personal client
Modify bank/branch
Group member inquiry
Agent appointment
NC-universal variable life
NC-hospital cash pPlan
NC-flexible premium
Medical provider inquiry
Policy loan inquiry
Modify allocation rules
Table inquire
Help lines inquiry
Policy notes inquiry
Reassurer inquiry
Modify client bank account
LIA record details inquiry
Test scenarios
Benchmark Testing Results: CSC’s Integral Life Insurance Solution and Microsoft SQL Server 15
Figure 7 shows that the average number of transactions per server stays nearly constant as the
number of servers, and the number of concurrent users, increases. The test with five application
servers and 1,500 users showed even higher average number of transactions.
Figure 7. Average number of transactions per server as the number of servers, and the number of concurrent users, increases
These results demonstrate that the solution provides excellent linear scalability: the response
times, as well as the average number of transactions per server, remain constant when adding
application servers up to five servers.
Application Server Metrics for Multiple Server Tests
Figure 8 visualizes the application server CPU and memory utilization and the total number of
transactions. Note that the average application server CPU utilization remained below 40 percent,
which is considered acceptable.
0
100
200
300
400
500
600
700
800
900
1 server,
200 users
2 servers,
400 users
3 servers,
600 users
4 servers,
800 users
5 servers,
1,000 users
5 servers,
1,500 users
Thro
ugh
pu
t (t
ran
sact
ion
s/se
rver
/min
ute
)
Benchmark Testing Results: CSC’s Integral Life Insurance Solution and Microsoft SQL Server 16
Figure 8. Multiple application server total number of transactions and CPU and memory utilizations
Database Server Metrics for Multiple Server Tests
Table 9 shows the CPU utilization by SQL Server and the number of batch requests per second
processed by SQL Server. The results show that the average database server CPU utilization
remained below 4.5 percent, with a maximum of only 4.71 percent for five application servers with
1,500 concurrent users. The average number of batch requests per second processed by SQL Server
was 9,786 for the five application servers with 1,500 users, with a maximum of 10,338 batch
requests per second.
Table 9. Database server metrics for multiple server tests
Database server metrics One app
server (200 users)
Two app servers
(400 users)
Three app servers
(600 users)
Four app servers
(800 users)
Five app servers (1,000 users)
Five app servers (1,500 users)
Average database server
CPU utilization 0.7% 1.3% 2.0% 2.7% 3.2% 4.3%
Maximum database
server CPU utilization 0.8% 1.4% 2.2% 3.0% 3.5% 4.7%
Average number of
batch requests/ second 1,521 2,960 4,442 5,940 7,377 9,786
Maximum number of
batch requests/ second 1,600 3,067 4,499 6,176 7,726 10,338
Figure 9 visualizes the database server metrics for the multiple server tests.
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
0
10
20
30
40
50
60
70
1 server,200 users
2 servers,400 users
3 servers,600 users
4 servers,800 users
5 servers,1,000 users
5 servers,1,500 users
Average CPUutilization (%)
Maximum CPUutilization (%)
Average memoryutilization (%)
Maximum memoryutilization (%)
Per
cen
t u
tiliz
atio
n
Tota
l th
rou
ghp
ut
(bu
sin
ess
tran
sact
ion
s/m
inu
te)
Benchmark Testing Results: CSC’s Integral Life Insurance Solution and Microsoft SQL Server 17
Figure 9. CPU utilization by SQL Server and the number of batch requests per second processed by SQL Server in the multiple server tests
Summary Scalability, stability, and performance are important to CSC insurance customers. These customers
want assurances that as their organizations grow and the number of policies increases, their
database software can maintain the performance they rely on. CSC has worked with Microsoft to
develop insurance solutions that combine CSC software with Microsoft products and technologies.
Benchmark testing and product optimizations are part of this joint effort.
The CSC and Microsoft benchmark tests were designed to simulate the real-world usage patterns of
large insurance customers. The test results were impressive. In the single application server test
with 330 concurrent users, the measured throughput was 811 business transactions per minute. For
the five application servers with 1,500 concurrent users test, the total throughput was 4,450
business transactions per minute.
SQL Server 2012 Enterprise performed remarkably well in the tests; for the five application servers
with 1,500 concurrent users, SQL Server processed an average 9,786 batch request per second (and
a maximum of 10,338 batch request per second), with an average of less than 5 percent CPU
utilization.
These results clearly confirm that CSC’s Integral running on SQL Server and Windows Server can
easily support at least 1,500 concurrent users. The results also show that the system is highly
scalable, with near-linear scalability as the number of application servers and the number of
concurrent users increase. CSC customers can select SQL Server as the database platform for large
and complex deployments of CSC’s Integral life insurance component with confidence.
0
2000
4000
6000
8000
10000
12000
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
1 app server,200 users
2 app servers,400 users
3 app servers,600 users
4 app servers,800 users
5 app servers,1,000 users
5 app servers,1,500 users
Average CPUutilization (%)
Maximum CPUutilization (%)
Average number ofbatch requests persecond
Maximum numberof batch requestsper second
Nu
mb
er o
f b
atch
req
ues
ts/s
eco
nd
Per
cen
t u
tiliz
atio
n
Benchmark Testing Results: CSC’s Integral Life Insurance Solution and Microsoft SQL Server 18
Additional Information The following references provide more information about CSC and Microsoft.
About CSC
CSC is a global leader in providing technology-enabled business solutions and services. For more
than 50 years, clients in industries and governments worldwide have trusted CSC with their
business process and information systems outsourcing, systems integration and consulting needs.
CSC has approximately 98,000 employees and reported revenue of $15.9 billion for the 12 months
ended March 30, 2012. The company trades on the New York Stock Exchange under the symbol
“CSC.”
For more information about CSC, visit www.csc.com.
About Microsoft
Founded in 1975, Microsoft (Nasdaq “MSFT”) is the worldwide leader in software, services, and
solutions that help people and businesses realize their full potential.
For more information about Microsoft products and services, visit www.microsoft.com.